[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:53:06 dist-test-slave-0b58 krb5kdc[31025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:53:06 dist-test-slave-0b58 krb5kdc[31025](info): set up 2 sockets
Aug 10 14:53:06 dist-test-slave-0b58 krb5kdc[31025](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 14:53:07.633762 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:53:07 dist-test-slave-0b58 krb5kdc[31025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:53:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:53:07.757122 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:37139
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:37139
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:07.839406 31041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:07.839627 31041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:07.839650 31041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:07.840869 31041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:07.840919 31041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:07.840932 31041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:07.840945 31041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:07.840957 31041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:07.842427 31041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:37139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:37139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:07.842621 31041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:07.842811 31041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:07.848075 31048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:07.848100 31041 server_base.cc:1034] running on GCE node
W20260810 14:53:07.848094 31046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:07.848081 31050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:07.848591 31041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:07.848881 31041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:07.850028 31041 hybrid_clock.cc:648] HybridClock initialized: now 1786373587850012 us; error 29 us; skew 500 ppm
Aug 10 14:53:07 dist-test-slave-0b58 krb5kdc[31025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373587, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:07.852577 31041 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:53:07.853083 31041 webserver.cc:460] Webserver started at http://127.30.74.126:36187/ using document root <none> and password file <none>
I20260810 14:53:07.853331 31041 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:07.853379 31041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:07.853543 31041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:07.854715 31041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "02dffda7dd3d4064b0241488b1f78d87"
format_stamp: "Formatted at 2026-08-10 14:53:07 on dist-test-slave-0b58"
server_key: "415b8857d3ea21c9c65d0ac58f7c83fb"
server_key_iv: "934833d0a370e8ab99b311027fa551ee"
server_key_version: "encryptionkey@0"
I20260810 14:53:07.855093 31041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "02dffda7dd3d4064b0241488b1f78d87"
format_stamp: "Formatted at 2026-08-10 14:53:07 on dist-test-slave-0b58"
server_key: "415b8857d3ea21c9c65d0ac58f7c83fb"
server_key_iv: "934833d0a370e8ab99b311027fa551ee"
server_key_version: "encryptionkey@0"
I20260810 14:53:07.856539 31041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260810 14:53:07.857362 31057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:07.857555 31041 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:07.857671 31041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "02dffda7dd3d4064b0241488b1f78d87"
format_stamp: "Formatted at 2026-08-10 14:53:07 on dist-test-slave-0b58"
server_key: "415b8857d3ea21c9c65d0ac58f7c83fb"
server_key_iv: "934833d0a370e8ab99b311027fa551ee"
server_key_version: "encryptionkey@0"
I20260810 14:53:07.857757 31041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:07.870977 31041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:07.872872 31041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:07.873032 31041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:07.877121 31041 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:37139
I20260810 14:53:07.877158 31111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:37139 every 8 connection(s)
I20260810 14:53:07.877503 31041 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:07.878376 31112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:07.880860 31112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87: Bootstrap starting.
I20260810 14:53:07.881471 31112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:07.881795 31112 log.cc:824] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:07.882611 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 31041
I20260810 14:53:07.882663 31112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87: No bootstrap required, opened a new log
I20260810 14:53:07.882726 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:07.883534 31017 external_mini_cluster.cc:1428] Setting key 6b71a27df9c00be3ec7720efa556a9d1
I20260810 14:53:07.885093 31112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02dffda7dd3d4064b0241488b1f78d87" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 37139 } }
I20260810 14:53:07.885232 31112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:07.885257 31112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02dffda7dd3d4064b0241488b1f78d87, State: Initialized, Role: FOLLOWER
I20260810 14:53:07.885430 31112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02dffda7dd3d4064b0241488b1f78d87" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 37139 } }
I20260810 14:53:07.885495 31112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:07.885519 31112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:07.885587 31112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:07.886445 31112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02dffda7dd3d4064b0241488b1f78d87" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 37139 } }
I20260810 14:53:07.886534 31112 leader_election.cc:302] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02dffda7dd3d4064b0241488b1f78d87; no voters:
I20260810 14:53:07.886711 31112 leader_election.cc:288] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:07.886760 31117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:07.886911 31117 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [term 1 LEADER]: Becoming Leader. State: Replica: 02dffda7dd3d4064b0241488b1f78d87, State: Running, Role: LEADER
I20260810 14:53:07.887006 31112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:07.887006 31117 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02dffda7dd3d4064b0241488b1f78d87" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 37139 } }
I20260810 14:53:07.887706 31118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02dffda7dd3d4064b0241488b1f78d87. Latest consensus state: current_term: 1 leader_uuid: "02dffda7dd3d4064b0241488b1f78d87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02dffda7dd3d4064b0241488b1f78d87" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 37139 } } }
I20260810 14:53:07.887801 31118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:07.888048 31126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:07.888163 31119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02dffda7dd3d4064b0241488b1f78d87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02dffda7dd3d4064b0241488b1f78d87" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 37139 } } }
I20260810 14:53:07.888273 31119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:07.888756 31126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:07.890475 31126 catalog_manager.cc:1347] Generated new cluster ID: e0beebcc961e46ec90c59d908b431450
I20260810 14:53:07.890532 31126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 14:53:07 dist-test-slave-0b58 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:07.899650 31116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:07.884574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33152 (local address 127.30.74.126:37139)
0810 14:53:07.884789 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:07.884800 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:07.884989 (+ 189us) server_negotiation.cc:407] Connection header received
0810 14:53:07.887056 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:07.887062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:07.887113 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:07.887239 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:07.889674 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:07.890910 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:07.892000 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:07.892502 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:07.892623 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:07.895984 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:07.895988 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:07.895988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:07.896006 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:07.897711 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:07.898172 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:07.898175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:07.898176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:07.898281 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:07.898546 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:07.898547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:07.898548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:07.898788 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:07.898967 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:07.899267 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 14:53:07.899402 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":99,"threads_started":1}
I20260810 14:53:07.900988 31126 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:07.901492 31126 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:07.906944 31126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02dffda7dd3d4064b0241488b1f78d87: Generated new TSK 0
I20260810 14:53:07.907123 31126 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:53:07.937004 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:37139
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:08.020638 31140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:08.020870 31140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:08.020895 31140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:08.022154 31140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:08.022202 31140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:08.022230 31140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:53:08.023710 31140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:37139
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:08.023972 31140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:08.024215 31140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:08.029232 31146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:08.029245 31148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:08.029237 31150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:08.029526 31140 server_base.cc:1034] running on GCE node
I20260810 14:53:08.029755 31140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:08.030038 31140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:08.031177 31140 hybrid_clock.cc:648] HybridClock initialized: now 1786373588031144 us; error 46 us; skew 500 ppm
Aug 10 14:53:08 dist-test-slave-0b58 krb5kdc[31025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373588, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:08.033803 31140 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:53:08.034225 31140 webserver.cc:460] Webserver started at http://127.30.74.65:33743/ using document root <none> and password file <none>
I20260810 14:53:08.034492 31140 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:08.034539 31140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:08.034708 31140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:08.035741 31140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "7929a47704eb4ddb9eb902d0806a33b5"
format_stamp: "Formatted at 2026-08-10 14:53:08 on dist-test-slave-0b58"
server_key: "d753f04685eb2757bbc4b684f77e3da1"
server_key_iv: "eea71c588481bd376f4a01c9c2facd9b"
server_key_version: "encryptionkey@0"
I20260810 14:53:08.036106 31140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "7929a47704eb4ddb9eb902d0806a33b5"
format_stamp: "Formatted at 2026-08-10 14:53:08 on dist-test-slave-0b58"
server_key: "d753f04685eb2757bbc4b684f77e3da1"
server_key_iv: "eea71c588481bd376f4a01c9c2facd9b"
server_key_version: "encryptionkey@0"
I20260810 14:53:08.037459 31140 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:08.038192 31157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:08.038379 31140 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 14:53:08.038471 31140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "7929a47704eb4ddb9eb902d0806a33b5"
format_stamp: "Formatted at 2026-08-10 14:53:08 on dist-test-slave-0b58"
server_key: "d753f04685eb2757bbc4b684f77e3da1"
server_key_iv: "eea71c588481bd376f4a01c9c2facd9b"
server_key_version: "encryptionkey@0"
I20260810 14:53:08.038544 31140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:08.054234 31140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:08.056222 31140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:08.056438 31140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:08.056677 31140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:08.056988 31140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:08.057024 31140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:08.057046 31140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:08.057090 31140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:08.063352 31140 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:43149
I20260810 14:53:08.063433 31281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:43149 every 8 connection(s)
I20260810 14:53:08.063759 31140 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
Aug 10 14:53:08 dist-test-slave-0b58 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373588, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:08.072747 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 31140
I20260810 14:53:08.072862 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:53:08.073024 31017 external_mini_cluster.cc:1428] Setting key fd79da6cafc10d7d91ee9caedd54178b
I20260810 14:53:08.073163 31116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:08.064630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:59437 (local address 127.30.74.126:37139)
0810 14:53:08.064703 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:08.064705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:08.064893 (+ 188us) server_negotiation.cc:407] Connection header received
0810 14:53:08.065356 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:08.065357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:08.065367 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:08.065424 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:08.066351 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:08.066862 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:08.067851 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:08.068047 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:08.068103 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:08.070364 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:08.070365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:08.070366 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:08.070373 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:08.071678 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:08.072045 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:08.072046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:08.072046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:08.072099 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:08.072368 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:08.072368 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:08.072368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:08.072478 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:08.072526 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:08.072926 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 14:53:08.073039 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 14:53:08.073302 31284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:08.064580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:37139 (local address 127.30.74.65:59437)
0810 14:53:08.064751 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:08.064796 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:08.065274 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:08.065476 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:08.065480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:08.066048 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:08.066267 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:08.066273 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:08.066944 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:08.066945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:08.067760 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:08.067763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:08.068128 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:08.068129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:08.068233 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:08.068782 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:08.068790 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:08.070237 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:08.071762 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:08.071768 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:08.071770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:08.071980 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:08.072158 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:08.072159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:08.072159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:08.072305 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:08.072533 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:08.072538 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:08.072728 (+ 190us) client_negotiation.cc:769] Sending connection context
0810 14:53:08.072856 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 14:53:08.073026 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260810 14:53:08.073930 31282 heartbeater.cc:347] Connected to a master server at 127.30.74.126:37139
I20260810 14:53:08.074020 31282 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:08.074251 31282 heartbeater.cc:511] Master 127.30.74.126:37139 requested a full tablet report, sending...
I20260810 14:53:08.074847 31075 ts_manager.cc:194] Registered new tserver with Master: 7929a47704eb4ddb9eb902d0806a33b5 (127.30.74.65:43149)
I20260810 14:53:08.075497 31075 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:59437
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:53:08.098621 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:37139
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:08.183648 31288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:08.183892 31288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:08.183920 31288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:08.185161 31288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:08.185211 31288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:08.185240 31288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:53:08.186762 31288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:37139
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:08.186968 31288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:08.187258 31288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:08.192615 31293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:08.192641 31294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:08.192741 31288 server_base.cc:1034] running on GCE node
W20260810 14:53:08.192844 31296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:08.193064 31288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:08.193388 31288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:08.194588 31288 hybrid_clock.cc:648] HybridClock initialized: now 1786373588194564 us; error 39 us; skew 500 ppm
Aug 10 14:53:08 dist-test-slave-0b58 krb5kdc[31025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373588, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:08.197211 31288 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:53:08.197698 31288 webserver.cc:460] Webserver started at http://127.30.74.66:35245/ using document root <none> and password file <none>
I20260810 14:53:08.197950 31288 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:08.197999 31288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:08.198173 31288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:08.199255 31288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "1f3a40c8a42c4098b3155dbb244b434c"
format_stamp: "Formatted at 2026-08-10 14:53:08 on dist-test-slave-0b58"
server_key: "f231d6d292674d39a41fe1171f91e13f"
server_key_iv: "c8e386e7a0a834caa559d8a030f07075"
server_key_version: "encryptionkey@0"
I20260810 14:53:08.199628 31288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "1f3a40c8a42c4098b3155dbb244b434c"
format_stamp: "Formatted at 2026-08-10 14:53:08 on dist-test-slave-0b58"
server_key: "f231d6d292674d39a41fe1171f91e13f"
server_key_iv: "c8e386e7a0a834caa559d8a030f07075"
server_key_version: "encryptionkey@0"
I20260810 14:53:08.201073 31288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 14:53:08.201860 31304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:08.202033 31288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:08.202171 31288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "1f3a40c8a42c4098b3155dbb244b434c"
format_stamp: "Formatted at 2026-08-10 14:53:08 on dist-test-slave-0b58"
server_key: "f231d6d292674d39a41fe1171f91e13f"
server_key_iv: "c8e386e7a0a834caa559d8a030f07075"
server_key_version: "encryptionkey@0"
I20260810 14:53:08.202252 31288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:08.221555 31288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:08.224443 31288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:08.224627 31288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:08.224882 31288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:08.225234 31288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:08.225296 31288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:08.225339 31288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:08.225376 31288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:08.233843 31288 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:43367
I20260810 14:53:08.233880 31427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:43367 every 8 connection(s)
I20260810 14:53:08.234232 31288 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
Aug 10 14:53:08 dist-test-slave-0b58 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373588, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:08.244347 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 31288
I20260810 14:53:08.244406 31116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:08.235062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:46901 (local address 127.30.74.126:37139)
0810 14:53:08.235160 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:08.235162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:08.235309 (+ 147us) server_negotiation.cc:407] Connection header received
0810 14:53:08.236513 (+ 1204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:08.236514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:08.236528 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:08.236581 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:08.237231 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:08.237742 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:08.238753 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:08.238973 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:08.239062 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:08.241522 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:08.241524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:08.241525 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:08.241533 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:08.242897 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:08.243330 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:08.243331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:08.243332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:08.243392 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:08.243663 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:08.243664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:08.243664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:08.243783 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:08.243856 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:08.244181 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 14:53:08.244306 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 14:53:08.244477 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:53:08.244544 31430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:08.234986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:37139 (local address 127.30.74.66:46901)
0810 14:53:08.235163 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:08.235219 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:08.236392 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:08.236656 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:08.236660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:08.236925 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:08.237128 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:08.237132 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:08.237844 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:08.237845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:08.238655 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:08.238658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:08.239073 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:08.239074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:08.239207 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:08.239791 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:08.239806 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:08.241420 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:08.242996 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:08.243004 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:08.243006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:08.243251 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:08.243471 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:08.243471 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:08.243472 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:08.243578 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:08.243860 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:08.243867 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:08.244008 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 14:53:08.244119 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 14:53:08.244312 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":70,"threads_started":1}
I20260810 14:53:08.244657 31017 external_mini_cluster.cc:1428] Setting key d81bfcf8b84d67138e35cb3d35bbcb15
I20260810 14:53:08.245026 31428 heartbeater.cc:347] Connected to a master server at 127.30.74.126:37139
I20260810 14:53:08.245151 31428 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:08.245342 31428 heartbeater.cc:511] Master 127.30.74.126:37139 requested a full tablet report, sending...
I20260810 14:53:08.245772 31075 ts_manager.cc:194] Registered new tserver with Master: 1f3a40c8a42c4098b3155dbb244b434c (127.30.74.66:43367)
I20260810 14:53:08.246302 31075 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:46901
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:53:08.271579 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:37139
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:08.356024 31433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:08.356232 31433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:08.356304 31433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:08.357576 31433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:08.357622 31433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:08.357649 31433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:53:08.359081 31433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:37139
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:08.359282 31433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:08.359468 31433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:08.364765 31439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:08.364790 31438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:08.364764 31441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:08.365051 31433 server_base.cc:1034] running on GCE node
I20260810 14:53:08.365262 31433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:08.365543 31433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:08.366722 31433 hybrid_clock.cc:648] HybridClock initialized: now 1786373588366705 us; error 33 us; skew 500 ppm
Aug 10 14:53:08 dist-test-slave-0b58 krb5kdc[31025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373588, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:08.369264 31433 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:53:08.369737 31433 webserver.cc:460] Webserver started at http://127.30.74.67:40675/ using document root <none> and password file <none>
I20260810 14:53:08.369984 31433 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:08.370059 31433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:08.370213 31433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:08.371313 31433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "ae2354f117684400820dc8f2c3af9f35"
format_stamp: "Formatted at 2026-08-10 14:53:08 on dist-test-slave-0b58"
server_key: "e98c27f29a3ca5edd99e9caa23c45fea"
server_key_iv: "5db5dbdebfb00d91507e8789181bffc3"
server_key_version: "encryptionkey@0"
I20260810 14:53:08.371685 31433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "ae2354f117684400820dc8f2c3af9f35"
format_stamp: "Formatted at 2026-08-10 14:53:08 on dist-test-slave-0b58"
server_key: "e98c27f29a3ca5edd99e9caa23c45fea"
server_key_iv: "5db5dbdebfb00d91507e8789181bffc3"
server_key_version: "encryptionkey@0"
I20260810 14:53:08.373168 31433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:53:08.373936 31448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:08.374150 31433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:08.374244 31433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "ae2354f117684400820dc8f2c3af9f35"
format_stamp: "Formatted at 2026-08-10 14:53:08 on dist-test-slave-0b58"
server_key: "e98c27f29a3ca5edd99e9caa23c45fea"
server_key_iv: "5db5dbdebfb00d91507e8789181bffc3"
server_key_version: "encryptionkey@0"
I20260810 14:53:08.374302 31433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:08.392633 31433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:08.395166 31433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:08.395294 31433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:08.395504 31433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:08.395856 31433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:08.395902 31433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:08.395925 31433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:08.395973 31433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:08.402462 31433 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:46825
I20260810 14:53:08.402544 31566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:46825 every 8 connection(s)
I20260810 14:53:08.402895 31433 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:53:08.407377 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 31433
I20260810 14:53:08.407491 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:53:08.407701 31017 external_mini_cluster.cc:1428] Setting key c3a60dd8b0168fc7f3b4b68009ee75c0
Aug 10 14:53:08 dist-test-slave-0b58 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373588, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:08.413178 31116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:08.403690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:48239 (local address 127.30.74.126:37139)
0810 14:53:08.403762 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:08.403768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:08.403986 (+ 218us) server_negotiation.cc:407] Connection header received
0810 14:53:08.404596 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:08.404597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:08.404608 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:08.404671 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:08.405700 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:08.406282 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:08.407583 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:08.407855 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:08.407931 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:08.410515 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:08.410516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:08.410516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:08.410524 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:08.411759 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:08.412238 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:08.412239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:08.412239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:08.412353 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:08.412632 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:08.412632 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:08.412633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:08.412747 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:08.412800 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:08.413049 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 14:53:08.413112 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 14:53:08.413434 31569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:08.403657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:37139 (local address 127.30.74.67:48239)
0810 14:53:08.403841 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:08.403884 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:08.404505 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:08.404732 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:08.404735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:08.405402 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:08.405608 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:08.405612 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:08.406364 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:08.406365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:08.407401 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:08.407404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:08.407957 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:08.407959 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:08.408166 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:08.408847 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:08.408858 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:08.410397 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:08.411866 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:08.411872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:08.411874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:08.412104 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:08.412442 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:08.412444 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:08.412444 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:08.412564 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:08.412808 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:08.412813 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:08.412936 (+ 123us) client_negotiation.cc:769] Sending connection context
0810 14:53:08.413023 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 14:53:08.413180 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":109,"threads_started":1}
I20260810 14:53:08.413966 31567 heartbeater.cc:347] Connected to a master server at 127.30.74.126:37139
I20260810 14:53:08.414062 31567 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:08.414275 31567 heartbeater.cc:511] Master 127.30.74.126:37139 requested a full tablet report, sending...
I20260810 14:53:08.414711 31075 ts_manager.cc:194] Registered new tserver with Master: ae2354f117684400820dc8f2c3af9f35 (127.30.74.67:46825)
I20260810 14:53:08.415089 31075 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:48239
I20260810 14:53:08.420065 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:53:08 dist-test-slave-0b58 krb5kdc[31025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373588, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 14:53:08 dist-test-slave-0b58 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373588, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.74.65@KRBTEST.COM
I20260810 14:53:08.444418 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:08.436826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45158 (local address 127.30.74.65:43149)
0810 14:53:08.437028 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:08.437033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:08.437045 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:53:08.437072 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:08.437074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:08.437089 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:08.437159 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:08.437715 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:08.438462 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:08.439159 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:08.439477 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:08.439534 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:08.441888 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:08.441896 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:08.441896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:08.441902 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:08.443302 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:08.443621 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:08.443624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:08.443625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:08.443682 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:08.443849 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:08.443850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:08.443850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:08.443977 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:08.444109 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:08.444147 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 14:53:08.444210 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
W20260810 14:53:08.444865 31188 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45158
Aug 10 14:53:08 dist-test-slave-0b58 krb5kdc[31025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 14:53:08 dist-test-slave-0b58 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.65@KRBTEST.COM
I20260810 14:53:08.467998 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:08.461242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45166 (local address 127.30.74.65:43149)
0810 14:53:08.461322 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:08.461324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:08.461367 (+ 43us) server_negotiation.cc:407] Connection header received
0810 14:53:08.461474 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:08.461475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:08.461489 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:08.461543 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:08.462111 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:08.462623 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:08.463329 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:08.463522 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:08.463574 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:08.465835 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:08.465836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:08.465836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:08.465843 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:08.467039 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:08.467344 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:08.467345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:08.467345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:08.467395 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:08.467564 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:08.467564 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:08.467565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:08.467653 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:08.467740 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:08.467822 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 14:53:08.467884 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 14:53:08.468590 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 31140
I20260810 14:53:08.475648 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 31288
I20260810 14:53:08.485872 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 31433
I20260810 14:53:08.491773 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 31041
2026-08-10T14:53:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1974 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:53:08 dist-test-slave-0b58 krb5kdc[31583](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:53:08 dist-test-slave-0b58 krb5kdc[31583](info): set up 2 sockets
Aug 10 14:53:08 dist-test-slave-0b58 krb5kdc[31583](info): commencing operation
krb5kdc: starting...
W20260810 14:53:09.566939 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:53:09 dist-test-slave-0b58 krb5kdc[31583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:53:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:53:09.688395 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:39405
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:43203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:39405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:09.770447 31597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:09.770648 31597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:09.770669 31597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:09.771736 31597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:09.771771 31597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:09.771786 31597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:09.771799 31597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:09.771811 31597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:09.773228 31597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:43203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:39405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:39405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:09.773430 31597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:09.773628 31597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:09.778527 31604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:09.778499 31603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:09.778512 31606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:09.778710 31597 server_base.cc:1034] running on GCE node
I20260810 14:53:09.778937 31597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:09.779292 31597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:09.780454 31597 hybrid_clock.cc:648] HybridClock initialized: now 1786373589780430 us; error 37 us; skew 500 ppm
Aug 10 14:53:09 dist-test-slave-0b58 krb5kdc[31583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373589, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:09.782956 31597 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:53:09.783392 31597 webserver.cc:460] Webserver started at http://127.30.74.126:44251/ using document root <none> and password file <none>
I20260810 14:53:09.783596 31597 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:09.783636 31597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:09.783731 31597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:09.784780 31597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "681cb2f845c545ea806c34a4cbc28b98"
format_stamp: "Formatted at 2026-08-10 14:53:09 on dist-test-slave-0b58"
server_key: "5b9e2c1ee32a8b5d3e350764f062b5f5"
server_key_iv: "871a7272695e9de2228ba49d9c68aadf"
server_key_version: "encryptionkey@0"
I20260810 14:53:09.785086 31597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "681cb2f845c545ea806c34a4cbc28b98"
format_stamp: "Formatted at 2026-08-10 14:53:09 on dist-test-slave-0b58"
server_key: "5b9e2c1ee32a8b5d3e350764f062b5f5"
server_key_iv: "871a7272695e9de2228ba49d9c68aadf"
server_key_version: "encryptionkey@0"
I20260810 14:53:09.786509 31597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:53:09.787261 31613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:09.787474 31597 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:09.787575 31597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "681cb2f845c545ea806c34a4cbc28b98"
format_stamp: "Formatted at 2026-08-10 14:53:09 on dist-test-slave-0b58"
server_key: "5b9e2c1ee32a8b5d3e350764f062b5f5"
server_key_iv: "871a7272695e9de2228ba49d9c68aadf"
server_key_version: "encryptionkey@0"
I20260810 14:53:09.787685 31597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:09.823099 31597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:09.825911 31597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:09.826079 31597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:09.830392 31597 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:39405
I20260810 14:53:09.830428 31668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:39405 every 8 connection(s)
I20260810 14:53:09.830808 31597 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:09.831712 31669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:09.834105 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 31597
I20260810 14:53:09.834110 31669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98: Bootstrap starting.
I20260810 14:53:09.834179 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:09.834348 31017 external_mini_cluster.cc:1428] Setting key 71b40634c900a177141f2d4eda489fdf
I20260810 14:53:09.834893 31669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:09.835242 31669 log.cc:824] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:09.836018 31669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98: No bootstrap required, opened a new log
I20260810 14:53:09.838611 31669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681cb2f845c545ea806c34a4cbc28b98" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39405 } }
I20260810 14:53:09.838760 31669 raft_consensus.cc:374] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:09.838784 31669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 681cb2f845c545ea806c34a4cbc28b98, State: Initialized, Role: FOLLOWER
I20260810 14:53:09.838922 31669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681cb2f845c545ea806c34a4cbc28b98" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39405 } }
I20260810 14:53:09.839004 31669 raft_consensus.cc:388] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:09.839048 31669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:09.839097 31669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:09.839814 31669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681cb2f845c545ea806c34a4cbc28b98" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39405 } }
I20260810 14:53:09.839926 31669 leader_election.cc:302] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 681cb2f845c545ea806c34a4cbc28b98; no voters:
I20260810 14:53:09.840107 31669 leader_election.cc:288] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:09.840178 31675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:09.840549 31669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:09.840459 31675 raft_consensus.cc:686] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [term 1 LEADER]: Becoming Leader. State: Replica: 681cb2f845c545ea806c34a4cbc28b98, State: Running, Role: LEADER
I20260810 14:53:09.840960 31675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681cb2f845c545ea806c34a4cbc28b98" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39405 } }
I20260810 14:53:09.841363 31676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "681cb2f845c545ea806c34a4cbc28b98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681cb2f845c545ea806c34a4cbc28b98" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39405 } } }
I20260810 14:53:09.841459 31676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:09.841352 31675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 681cb2f845c545ea806c34a4cbc28b98. Latest consensus state: current_term: 1 leader_uuid: "681cb2f845c545ea806c34a4cbc28b98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "681cb2f845c545ea806c34a4cbc28b98" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39405 } } }
I20260810 14:53:09.841688 31685 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:09.842404 31685 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:09.842985 31675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:09.843876 31685 catalog_manager.cc:1347] Generated new cluster ID: dcc487deeeac4ab8b3c9a354303a4f9e
I20260810 14:53:09.843935 31685 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 14:53:09 dist-test-slave-0b58 krb5kdc[31583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:09.849493 31685 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:09.850205 31685 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:09.851040 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:09.834934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58676 (local address 127.30.74.126:39405)
0810 14:53:09.835151 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:09.835173 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:09.835212 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:53:09.835796 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:09.835805 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:09.835868 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:09.836032 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:09.836741 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:09.838416 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:09.839432 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:09.840092 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:09.840215 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:09.847138 (+ 6923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:09.847143 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:09.847143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:09.847156 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:09.849328 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:09.849789 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:09.849793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:09.849793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:09.849891 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:09.850137 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:09.850138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:09.850138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:09.850417 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:09.850598 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:09.850670 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 14:53:09.850838 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":92,"threads_started":1}
I20260810 14:53:09.858074 31685 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 681cb2f845c545ea806c34a4cbc28b98: Generated new TSK 0
I20260810 14:53:09.858249 31685 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:53:09.890687 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39405
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:43203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:09.976177 31699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:09.976441 31699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:09.976468 31699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:09.977586 31699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:09.977620 31699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:09.977646 31699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:53:09.979058 31699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:43203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39405
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:09.979255 31699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:09.979427 31699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:09.984421 31705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:09.984400 31704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:09.984400 31707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:09.984711 31699 server_base.cc:1034] running on GCE node
I20260810 14:53:09.984952 31699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:09.985214 31699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:09.986346 31699 hybrid_clock.cc:648] HybridClock initialized: now 1786373589986334 us; error 26 us; skew 500 ppm
Aug 10 14:53:09 dist-test-slave-0b58 krb5kdc[31583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373589, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:09.989090 31699 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:53:09.989552 31699 webserver.cc:460] Webserver started at http://127.30.74.65:42047/ using document root <none> and password file <none>
I20260810 14:53:09.989817 31699 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:09.989889 31699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:09.990044 31699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:09.991169 31699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "9651596504d149e79d18779ca8f85d6f"
format_stamp: "Formatted at 2026-08-10 14:53:09 on dist-test-slave-0b58"
server_key: "0f56b2c517ede87c0b78cff4d77970a6"
server_key_iv: "f368b11a2364b24b3c359d1829992b17"
server_key_version: "encryptionkey@0"
I20260810 14:53:09.991592 31699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "9651596504d149e79d18779ca8f85d6f"
format_stamp: "Formatted at 2026-08-10 14:53:09 on dist-test-slave-0b58"
server_key: "0f56b2c517ede87c0b78cff4d77970a6"
server_key_iv: "f368b11a2364b24b3c359d1829992b17"
server_key_version: "encryptionkey@0"
I20260810 14:53:09.993062 31699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:53:09.993948 31714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:09.994184 31699 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:09.994314 31699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "9651596504d149e79d18779ca8f85d6f"
format_stamp: "Formatted at 2026-08-10 14:53:09 on dist-test-slave-0b58"
server_key: "0f56b2c517ede87c0b78cff4d77970a6"
server_key_iv: "f368b11a2364b24b3c359d1829992b17"
server_key_version: "encryptionkey@0"
I20260810 14:53:09.994390 31699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:10.010417 31699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:10.012543 31699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:10.012729 31699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:10.012962 31699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:10.013283 31699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:10.013355 31699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:10.013393 31699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:10.013407 31699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:10.019332 31699 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:40607
I20260810 14:53:10.019390 31834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:40607 every 8 connection(s)
I20260810 14:53:10.019729 31699 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
Aug 10 14:53:10 dist-test-slave-0b58 krb5kdc[31583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373589, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:10.026867 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 31699
I20260810 14:53:10.026971 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:53:10.027134 31017 external_mini_cluster.cc:1428] Setting key 257c98ef3dc7c2562152e5defd535a8c
I20260810 14:53:10.029842 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.020460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:34087 (local address 127.30.74.126:39405)
0810 14:53:10.020504 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:10.020507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:10.020705 (+ 198us) server_negotiation.cc:407] Connection header received
0810 14:53:10.021698 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:10.021700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:10.021712 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:10.021793 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:10.022461 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.022977 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.024008 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.024217 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.024290 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.026475 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:10.026477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:10.026477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:10.026485 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:10.028171 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:10.028725 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:10.028726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:10.028726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:10.028785 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:10.029072 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:10.029072 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:10.029073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:10.029183 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:10.029252 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:10.029582 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 14:53:10.029709 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 14:53:10.030036 31837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.020402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39405 (local address 127.30.74.65:34087)
0810 14:53:10.020570 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:10.020619 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:10.021559 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:10.021881 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:10.021885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:10.022172 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:10.022367 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:10.022370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.023067 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.023068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:10.023928 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:10.023931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.024346 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.024348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:10.024470 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.024962 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:10.024971 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:10.026368 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:10.028298 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:10.028306 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:10.028308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:10.028632 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:10.028856 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:10.028856 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:10.028857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:10.029003 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:10.029262 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:10.029268 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:10.029438 (+ 170us) client_negotiation.cc:769] Sending connection context
0810 14:53:10.029554 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 14:53:10.029766 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260810 14:53:10.030764 31835 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39405
I20260810 14:53:10.030876 31835 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:10.031057 31835 heartbeater.cc:511] Master 127.30.74.126:39405 requested a full tablet report, sending...
I20260810 14:53:10.031626 31631 ts_manager.cc:194] Registered new tserver with Master: 9651596504d149e79d18779ca8f85d6f (127.30.74.65:40607)
I20260810 14:53:10.032366 31631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:34087
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:53:10.054811 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39405
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:43203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:10.137935 31840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:10.138172 31840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:10.138216 31840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:10.139376 31840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:10.139442 31840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:10.139488 31840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:53:10.141152 31840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:43203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39405
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:10.141451 31840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:10.141683 31840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:10.146672 31847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:10.146677 31849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:10.146754 31846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:10.147023 31840 server_base.cc:1034] running on GCE node
I20260810 14:53:10.147248 31840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:10.147465 31840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:10.148641 31840 hybrid_clock.cc:648] HybridClock initialized: now 1786373590148632 us; error 38 us; skew 500 ppm
Aug 10 14:53:10 dist-test-slave-0b58 krb5kdc[31583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373590, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:10.151294 31840 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:53:10.151858 31840 webserver.cc:460] Webserver started at http://127.30.74.66:33343/ using document root <none> and password file <none>
I20260810 14:53:10.152141 31840 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:10.152191 31840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:10.152354 31840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:10.153479 31840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "aa7de893827e4d01a88d53667cb69818"
format_stamp: "Formatted at 2026-08-10 14:53:10 on dist-test-slave-0b58"
server_key: "2ba69aa7d395791c88040784e3b2c267"
server_key_iv: "d0204ebad998fe3f932283b3557c4574"
server_key_version: "encryptionkey@0"
I20260810 14:53:10.153843 31840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "aa7de893827e4d01a88d53667cb69818"
format_stamp: "Formatted at 2026-08-10 14:53:10 on dist-test-slave-0b58"
server_key: "2ba69aa7d395791c88040784e3b2c267"
server_key_iv: "d0204ebad998fe3f932283b3557c4574"
server_key_version: "encryptionkey@0"
I20260810 14:53:10.155198 31840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:53:10.155941 31857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:10.156149 31840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:10.156294 31840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "aa7de893827e4d01a88d53667cb69818"
format_stamp: "Formatted at 2026-08-10 14:53:10 on dist-test-slave-0b58"
server_key: "2ba69aa7d395791c88040784e3b2c267"
server_key_iv: "d0204ebad998fe3f932283b3557c4574"
server_key_version: "encryptionkey@0"
I20260810 14:53:10.156388 31840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:10.170465 31840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:10.172821 31840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:10.172992 31840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:10.173231 31840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:10.173550 31840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:10.173605 31840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:10.173646 31840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:10.173674 31840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:10.180030 31840 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:33375
I20260810 14:53:10.180136 31981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:33375 every 8 connection(s)
I20260810 14:53:10.180522 31840 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
Aug 10 14:53:10 dist-test-slave-0b58 krb5kdc[31583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373590, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:10.190387 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.181314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:50201 (local address 127.30.74.126:39405)
0810 14:53:10.181380 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:10.181382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:10.181588 (+ 206us) server_negotiation.cc:407] Connection header received
0810 14:53:10.182601 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:10.182603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:10.182617 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:10.182676 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:10.183317 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.183834 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.184946 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.185154 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.185242 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.187462 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:10.187463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:10.187464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:10.187471 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:10.188974 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:10.189430 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:10.189431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:10.189431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:10.189483 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:10.189738 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:10.189739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:10.189739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:10.189902 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:10.189956 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:10.190231 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 14:53:10.190320 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 14:53:10.190466 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 31840
I20260810 14:53:10.190557 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:53:10.190749 31017 external_mini_cluster.cc:1428] Setting key 018cb08df9bf5336a22e2daec998e84d
I20260810 14:53:10.190747 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.181303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39405 (local address 127.30.74.66:50201)
0810 14:53:10.181469 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:10.181503 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:10.182476 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:10.182753 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:10.182756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:10.183011 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:10.183208 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:10.183213 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.183939 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.183940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:10.184840 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:10.184843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.185255 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.185257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:10.185383 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.185861 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:10.185870 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:10.187362 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:10.189072 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:10.189079 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:10.189081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:10.189352 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:10.189547 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:10.189548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:10.189548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:10.189665 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:10.189966 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:10.189972 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:10.190105 (+ 133us) client_negotiation.cc:769] Sending connection context
0810 14:53:10.190232 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 14:53:10.190433 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260810 14:53:10.191367 31982 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39405
I20260810 14:53:10.191468 31982 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:10.191680 31982 heartbeater.cc:511] Master 127.30.74.126:39405 requested a full tablet report, sending...
I20260810 14:53:10.192152 31630 ts_manager.cc:194] Registered new tserver with Master: aa7de893827e4d01a88d53667cb69818 (127.30.74.66:33375)
I20260810 14:53:10.192565 31630 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:50201
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:53:10.217269 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39405
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:43203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:10.312690 31987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:10.312922 31987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:10.312949 31987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:10.314309 31987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:10.314358 31987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:10.314389 31987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:53:10.316080 31987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:43203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39405
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:10.316370 31987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:10.316649 31987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:10.322077 31995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:10.322072 31993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:10.322311 31987 server_base.cc:1034] running on GCE node
W20260810 14:53:10.322077 31992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:10.322664 31987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:10.322953 31987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:10.324152 31987 hybrid_clock.cc:648] HybridClock initialized: now 1786373590324148 us; error 40 us; skew 500 ppm
Aug 10 14:53:10 dist-test-slave-0b58 krb5kdc[31583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373590, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:10.326958 31987 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:53:10.327464 31987 webserver.cc:460] Webserver started at http://127.30.74.67:33505/ using document root <none> and password file <none>
I20260810 14:53:10.327718 31987 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:10.327800 31987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:10.327962 31987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:10.329219 31987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "819b49bace174d8bb08be94b1e699261"
format_stamp: "Formatted at 2026-08-10 14:53:10 on dist-test-slave-0b58"
server_key: "42b3e37594090b2bdf91b38b2f85b9c2"
server_key_iv: "949c3215eb718ae4e8a008e516de9afc"
server_key_version: "encryptionkey@0"
I20260810 14:53:10.329645 31987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "819b49bace174d8bb08be94b1e699261"
format_stamp: "Formatted at 2026-08-10 14:53:10 on dist-test-slave-0b58"
server_key: "42b3e37594090b2bdf91b38b2f85b9c2"
server_key_iv: "949c3215eb718ae4e8a008e516de9afc"
server_key_version: "encryptionkey@0"
I20260810 14:53:10.331128 31987 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 14:53:10.331940 32002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:10.332125 31987 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:10.332247 31987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "819b49bace174d8bb08be94b1e699261"
format_stamp: "Formatted at 2026-08-10 14:53:10 on dist-test-slave-0b58"
server_key: "42b3e37594090b2bdf91b38b2f85b9c2"
server_key_iv: "949c3215eb718ae4e8a008e516de9afc"
server_key_version: "encryptionkey@0"
I20260810 14:53:10.332398 31987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:10.350602 31987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:10.353555 31987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:10.353693 31987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:10.353921 31987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:10.354290 31987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:10.354354 31987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:10.354401 31987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:10.354435 31987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:10.361616 31987 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:41905
I20260810 14:53:10.361754 32121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:41905 every 8 connection(s)
I20260810 14:53:10.362080 31987 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:53:10.363026 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 31987
I20260810 14:53:10.363134 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:53:10.363317 31017 external_mini_cluster.cc:1428] Setting key 6899c95fbe232101f5bb99a105af93e8
Aug 10 14:53:10 dist-test-slave-0b58 krb5kdc[31583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373590, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:10.372468 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.363011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:35531 (local address 127.30.74.126:39405)
0810 14:53:10.363075 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:10.363078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:10.363414 (+ 336us) server_negotiation.cc:407] Connection header received
0810 14:53:10.364685 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:10.364686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:10.364698 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:10.364749 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:10.365352 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.365893 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.366959 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.367163 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.367229 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.369611 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:10.369612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:10.369613 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:10.369620 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:10.371169 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:10.371557 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:10.371558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:10.371558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:10.371615 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:10.371840 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:10.371840 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:10.371840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:10.371953 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:10.372008 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:10.372328 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 14:53:10.372396 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 14:53:10.372702 32124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.362960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39405 (local address 127.30.74.67:35531)
0810 14:53:10.363245 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:10.363298 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:10.364589 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:10.364804 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:10.364807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:10.365074 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:10.365278 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:10.365282 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.366006 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.366007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:10.366862 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:10.366864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.367250 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.367251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:10.367359 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.367876 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:10.367884 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:10.369508 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:10.371255 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:10.371262 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:10.371264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:10.371485 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:10.371678 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:10.371679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:10.371679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:10.371780 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:10.372011 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:10.372016 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:10.372130 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 14:53:10.372240 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 14:53:10.372440 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":98,"threads_started":1}
I20260810 14:53:10.373263 32122 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39405
I20260810 14:53:10.373373 32122 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:10.373579 32122 heartbeater.cc:511] Master 127.30.74.126:39405 requested a full tablet report, sending...
I20260810 14:53:10.374006 31630 ts_manager.cc:194] Registered new tserver with Master: 819b49bace174d8bb08be94b1e699261 (127.30.74.67:41905)
I20260810 14:53:10.374439 31630 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:35531
I20260810 14:53:10.375219 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:53:10.382932 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.376881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58680 (local address 127.30.74.126:39405)
0810 14:53:10.376950 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:10.376952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:10.377002 (+ 50us) server_negotiation.cc:407] Connection header received
0810 14:53:10.377044 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:10.377045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:10.377053 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:10.377108 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:10.377657 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.378197 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.378919 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.379118 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.379183 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.380379 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:10.380380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:10.380381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:10.380387 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:10.381854 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:10.382227 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:10.382228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:10.382228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:10.382295 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:10.382465 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:10.382466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:10.382466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:10.382607 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:10.382689 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:10.382777 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 14:53:10.382838 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 14:53:10.385354 31631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:53:10.387064 31631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:10.394723 32137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.390143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47354 (local address 127.30.74.65:40607)
0810 14:53:10.390505 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:10.390511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:10.390874 (+ 363us) server_negotiation.cc:407] Connection header received
0810 14:53:10.390923 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:10.390926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:10.390941 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:10.391020 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:10.391240 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.392448 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.393766 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.394262 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.394353 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.394416 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:10.394579 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 14:53:10.394621 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":95,"threads_started":1}
I20260810 14:53:10.394762 32136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.389896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:40607 (local address 127.0.0.1:47354)
0810 14:53:10.390559 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:10.390571 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:10.390635 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:10.391114 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:10.391115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:10.391118 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:10.391161 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:10.391163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.392606 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.392607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:10.393678 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:10.393680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.394354 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.394355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:10.394441 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.394456 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 14:53:10.394560 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 14:53:10.394619 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"mutex_wait_us":53,"thread_start_us":80,"threads_started":1}
I20260810 14:53:10.395120 32138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.390252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42770 (local address 127.30.74.66:33375)
0810 14:53:10.390461 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:10.390482 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:10.390760 (+ 278us) server_negotiation.cc:407] Connection header received
0810 14:53:10.390810 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:10.390812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:10.390828 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:10.390909 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:10.391142 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.392449 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.393828 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.394610 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.394675 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.394733 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:10.394963 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 14:53:10.395008 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":117,"threads_started":1}
I20260810 14:53:10.395253 32135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.390011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42692 (local address 127.30.74.67:41905)
0810 14:53:10.390249 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:10.390254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:10.390507 (+ 253us) server_negotiation.cc:407] Connection header received
0810 14:53:10.390609 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:10.390627 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:10.390646 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:10.390751 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:10.391010 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.391883 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.393459 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.394275 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.394353 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.394416 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:10.394817 (+ 401us) server_negotiation.cc:299] Negotiation successful
0810 14:53:10.394869 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":112,"threads_started":1}
I20260810 14:53:10.395074 32134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.390126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:33375 (local address 127.0.0.1:42770)
0810 14:53:10.390640 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:10.390649 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:10.390698 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:10.391006 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:10.391007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:10.391010 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:10.391052 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:10.391053 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.392592 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.392593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:10.393708 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:10.393711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.394796 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.394797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:10.394875 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.394880 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:53:10.394942 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 14:53:10.394994 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":34,"thread_start_us":111,"threads_started":1}
I20260810 14:53:10.394771 32133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.389896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:41905 (local address 127.0.0.1:42692)
0810 14:53:10.390362 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:10.390410 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:10.390503 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:10.390752 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:10.390755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:10.390757 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:10.390896 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:10.390898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.391995 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.391997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:10.393330 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:10.393332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.394458 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.394459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:10.394530 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.394535 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:53:10.394591 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 14:53:10.394646 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":79,"threads_started":1}
I20260810 14:53:10.396167 31765 tablet_service.cc:1467] Processing CreateTablet for tablet d7f5fce557784c28b7836b69ff1565a4 (DEFAULT_TABLE table=test-table [id=3ecaebb1a2bf4d4db7dd533e2a37f04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:10.396173 31909 tablet_service.cc:1467] Processing CreateTablet for tablet d7f5fce557784c28b7836b69ff1565a4 (DEFAULT_TABLE table=test-table [id=3ecaebb1a2bf4d4db7dd533e2a37f04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:10.396634 31765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7f5fce557784c28b7836b69ff1565a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:10.396664 31909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7f5fce557784c28b7836b69ff1565a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:10.396981 32054 tablet_service.cc:1467] Processing CreateTablet for tablet d7f5fce557784c28b7836b69ff1565a4 (DEFAULT_TABLE table=test-table [id=3ecaebb1a2bf4d4db7dd533e2a37f04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:10.397331 32054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7f5fce557784c28b7836b69ff1565a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:10.399518 32139 tablet_bootstrap.cc:492] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f: Bootstrap starting.
I20260810 14:53:10.400130 32139 tablet_bootstrap.cc:654] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:10.400674 32140 tablet_bootstrap.cc:492] T d7f5fce557784c28b7836b69ff1565a4 P aa7de893827e4d01a88d53667cb69818: Bootstrap starting.
I20260810 14:53:10.400851 32139 log.cc:824] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:10.401418 32140 tablet_bootstrap.cc:654] T d7f5fce557784c28b7836b69ff1565a4 P aa7de893827e4d01a88d53667cb69818: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:10.401401 32141 tablet_bootstrap.cc:492] T d7f5fce557784c28b7836b69ff1565a4 P 819b49bace174d8bb08be94b1e699261: Bootstrap starting.
I20260810 14:53:10.401809 32139 tablet_bootstrap.cc:492] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f: No bootstrap required, opened a new log
I20260810 14:53:10.401901 32139 ts_tablet_manager.cc:1397] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:53:10.402070 32140 log.cc:824] T d7f5fce557784c28b7836b69ff1565a4 P aa7de893827e4d01a88d53667cb69818: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:10.402168 32141 tablet_bootstrap.cc:654] T d7f5fce557784c28b7836b69ff1565a4 P 819b49bace174d8bb08be94b1e699261: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:10.402736 32141 log.cc:824] T d7f5fce557784c28b7836b69ff1565a4 P 819b49bace174d8bb08be94b1e699261: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:10.403033 32140 tablet_bootstrap.cc:492] T d7f5fce557784c28b7836b69ff1565a4 P aa7de893827e4d01a88d53667cb69818: No bootstrap required, opened a new log
I20260810 14:53:10.403106 32140 ts_tablet_manager.cc:1397] T d7f5fce557784c28b7836b69ff1565a4 P aa7de893827e4d01a88d53667cb69818: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:53:10.403748 32141 tablet_bootstrap.cc:492] T d7f5fce557784c28b7836b69ff1565a4 P 819b49bace174d8bb08be94b1e699261: No bootstrap required, opened a new log
I20260810 14:53:10.403834 32141 ts_tablet_manager.cc:1397] T d7f5fce557784c28b7836b69ff1565a4 P 819b49bace174d8bb08be94b1e699261: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:53:10.403944 32139 raft_consensus.cc:348] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9651596504d149e79d18779ca8f85d6f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40607 } } peers { permanent_uuid: "819b49bace174d8bb08be94b1e699261" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 41905 } } peers { permanent_uuid: "aa7de893827e4d01a88d53667cb69818" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33375 } }
I20260810 14:53:10.404120 32139 raft_consensus.cc:374] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:10.404165 32139 raft_consensus.cc:729] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9651596504d149e79d18779ca8f85d6f, State: Initialized, Role: FOLLOWER
I20260810 14:53:10.404338 32139 consensus_queue.cc:260] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9651596504d149e79d18779ca8f85d6f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40607 } } peers { permanent_uuid: "819b49bace174d8bb08be94b1e699261" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 41905 } } peers { permanent_uuid: "aa7de893827e4d01a88d53667cb69818" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33375 } }
I20260810 14:53:10.404652 32139 ts_tablet_manager.cc:1428] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:53:10.404803 31835 heartbeater.cc:503] Master 127.30.74.126:39405 was elected leader, sending a full tablet report...
I20260810 14:53:10.405800 32140 raft_consensus.cc:348] T d7f5fce557784c28b7836b69ff1565a4 P aa7de893827e4d01a88d53667cb69818 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9651596504d149e79d18779ca8f85d6f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40607 } } peers { permanent_uuid: "819b49bace174d8bb08be94b1e699261" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 41905 } } peers { permanent_uuid: "aa7de893827e4d01a88d53667cb69818" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33375 } }
I20260810 14:53:10.406005 32140 raft_consensus.cc:374] T d7f5fce557784c28b7836b69ff1565a4 P aa7de893827e4d01a88d53667cb69818 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:10.406051 32140 raft_consensus.cc:729] T d7f5fce557784c28b7836b69ff1565a4 P aa7de893827e4d01a88d53667cb69818 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa7de893827e4d01a88d53667cb69818, State: Initialized, Role: FOLLOWER
I20260810 14:53:10.406105 32141 raft_consensus.cc:348] T d7f5fce557784c28b7836b69ff1565a4 P 819b49bace174d8bb08be94b1e699261 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9651596504d149e79d18779ca8f85d6f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40607 } } peers { permanent_uuid: "819b49bace174d8bb08be94b1e699261" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 41905 } } peers { permanent_uuid: "aa7de893827e4d01a88d53667cb69818" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33375 } }
I20260810 14:53:10.406201 32140 consensus_queue.cc:260] T d7f5fce557784c28b7836b69ff1565a4 P aa7de893827e4d01a88d53667cb69818 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9651596504d149e79d18779ca8f85d6f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40607 } } peers { permanent_uuid: "819b49bace174d8bb08be94b1e699261" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 41905 } } peers { permanent_uuid: "aa7de893827e4d01a88d53667cb69818" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33375 } }
I20260810 14:53:10.406286 32141 raft_consensus.cc:374] T d7f5fce557784c28b7836b69ff1565a4 P 819b49bace174d8bb08be94b1e699261 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:10.406337 32141 raft_consensus.cc:729] T d7f5fce557784c28b7836b69ff1565a4 P 819b49bace174d8bb08be94b1e699261 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 819b49bace174d8bb08be94b1e699261, State: Initialized, Role: FOLLOWER
I20260810 14:53:10.406473 32141 consensus_queue.cc:260] T d7f5fce557784c28b7836b69ff1565a4 P 819b49bace174d8bb08be94b1e699261 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9651596504d149e79d18779ca8f85d6f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40607 } } peers { permanent_uuid: "819b49bace174d8bb08be94b1e699261" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 41905 } } peers { permanent_uuid: "aa7de893827e4d01a88d53667cb69818" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33375 } }
I20260810 14:53:10.406520 32140 ts_tablet_manager.cc:1428] T d7f5fce557784c28b7836b69ff1565a4 P aa7de893827e4d01a88d53667cb69818: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:53:10.406522 31982 heartbeater.cc:503] Master 127.30.74.126:39405 was elected leader, sending a full tablet report...
I20260810 14:53:10.406787 32141 ts_tablet_manager.cc:1428] T d7f5fce557784c28b7836b69ff1565a4 P 819b49bace174d8bb08be94b1e699261: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:53:10.406803 32122 heartbeater.cc:503] Master 127.30.74.126:39405 was elected leader, sending a full tablet report...
W20260810 14:53:10.431142 31983 tablet.cc:2369] T d7f5fce557784c28b7836b69ff1565a4 P aa7de893827e4d01a88d53667cb69818: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:53:10.520426 31836 tablet.cc:2369] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:53:10.528012 32145 raft_consensus.cc:482] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:10.528141 32145 raft_consensus.cc:504] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9651596504d149e79d18779ca8f85d6f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40607 } } peers { permanent_uuid: "819b49bace174d8bb08be94b1e699261" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 41905 } } peers { permanent_uuid: "aa7de893827e4d01a88d53667cb69818" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33375 } }
I20260810 14:53:10.528663 32145 leader_election.cc:288] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 819b49bace174d8bb08be94b1e699261 (127.30.74.67:41905), aa7de893827e4d01a88d53667cb69818 (127.30.74.66:33375)
I20260810 14:53:10.531864 31837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.528913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:41905 (local address 127.30.74.65:56491)
0810 14:53:10.528986 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:10.529000 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:10.529062 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:10.529231 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:10.529233 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:10.529236 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:10.529299 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:10.529301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.530067 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.530068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:10.530921 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:10.530924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.531578 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.531579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:10.531659 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.531663 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:10.531729 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 14:53:10.531770 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 14:53:10.531877 32135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.529020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:56491 (local address 127.30.74.67:41905)
0810 14:53:10.529080 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:10.529082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:10.529091 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:10.529125 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:10.529126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:10.529137 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:10.529190 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:10.529400 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.529967 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.531016 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.531480 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.531538 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.531545 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:10.531751 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 14:53:10.531784 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 14:53:10.532176 32138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.529119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:33541 (local address 127.30.74.66:33375)
0810 14:53:10.529183 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:10.529194 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:10.529240 (+ 46us) server_negotiation.cc:407] Connection header received
0810 14:53:10.529302 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:10.529303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:10.529316 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:10.529374 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:10.529634 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.530210 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.531259 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.531772 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.531847 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.531854 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:10.532063 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 14:53:10.532109 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 14:53:10.532344 32075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7f5fce557784c28b7836b69ff1565a4" candidate_uuid: "9651596504d149e79d18779ca8f85d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "819b49bace174d8bb08be94b1e699261" is_pre_election: true
I20260810 14:53:10.532442 32148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.529028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:33375 (local address 127.30.74.65:33541)
0810 14:53:10.529183 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:10.529192 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:10.529245 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:10.529417 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:10.529417 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:10.529419 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:10.529519 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:10.529521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.530315 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.530315 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:10.531142 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:10.531144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.531878 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.531879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:10.531955 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.531961 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:53:10.532306 (+ 345us) client_negotiation.cc:240] Negotiation successful
0810 14:53:10.532360 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260810 14:53:10.532557 32075 raft_consensus.cc:2454] T d7f5fce557784c28b7836b69ff1565a4 P 819b49bace174d8bb08be94b1e699261 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9651596504d149e79d18779ca8f85d6f in term 0.
I20260810 14:53:10.532831 31717 leader_election.cc:302] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 819b49bace174d8bb08be94b1e699261, 9651596504d149e79d18779ca8f85d6f; no voters:
I20260810 14:53:10.532835 31930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7f5fce557784c28b7836b69ff1565a4" candidate_uuid: "9651596504d149e79d18779ca8f85d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa7de893827e4d01a88d53667cb69818" is_pre_election: true
I20260810 14:53:10.532974 31930 raft_consensus.cc:2454] T d7f5fce557784c28b7836b69ff1565a4 P aa7de893827e4d01a88d53667cb69818 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9651596504d149e79d18779ca8f85d6f in term 0.
I20260810 14:53:10.532970 32145 raft_consensus.cc:2781] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:10.533025 32145 raft_consensus.cc:482] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:10.533080 32145 raft_consensus.cc:3037] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:10.533775 32145 raft_consensus.cc:504] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9651596504d149e79d18779ca8f85d6f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40607 } } peers { permanent_uuid: "819b49bace174d8bb08be94b1e699261" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 41905 } } peers { permanent_uuid: "aa7de893827e4d01a88d53667cb69818" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33375 } }
I20260810 14:53:10.533881 32145 leader_election.cc:288] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [CANDIDATE]: Term 1 election: Requested vote from peers 819b49bace174d8bb08be94b1e699261 (127.30.74.67:41905), aa7de893827e4d01a88d53667cb69818 (127.30.74.66:33375)
I20260810 14:53:10.534083 32075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7f5fce557784c28b7836b69ff1565a4" candidate_uuid: "9651596504d149e79d18779ca8f85d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "819b49bace174d8bb08be94b1e699261"
I20260810 14:53:10.534149 32075 raft_consensus.cc:3037] T d7f5fce557784c28b7836b69ff1565a4 P 819b49bace174d8bb08be94b1e699261 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:10.534157 31930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7f5fce557784c28b7836b69ff1565a4" candidate_uuid: "9651596504d149e79d18779ca8f85d6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa7de893827e4d01a88d53667cb69818"
I20260810 14:53:10.534240 31930 raft_consensus.cc:3037] T d7f5fce557784c28b7836b69ff1565a4 P aa7de893827e4d01a88d53667cb69818 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:10.534929 32075 raft_consensus.cc:2454] T d7f5fce557784c28b7836b69ff1565a4 P 819b49bace174d8bb08be94b1e699261 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9651596504d149e79d18779ca8f85d6f in term 1.
I20260810 14:53:10.535032 31930 raft_consensus.cc:2454] T d7f5fce557784c28b7836b69ff1565a4 P aa7de893827e4d01a88d53667cb69818 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9651596504d149e79d18779ca8f85d6f in term 1.
I20260810 14:53:10.535101 31717 leader_election.cc:302] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 819b49bace174d8bb08be94b1e699261, 9651596504d149e79d18779ca8f85d6f; no voters:
I20260810 14:53:10.535221 32145 raft_consensus.cc:2781] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:10.535427 32145 raft_consensus.cc:686] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [term 1 LEADER]: Becoming Leader. State: Replica: 9651596504d149e79d18779ca8f85d6f, State: Running, Role: LEADER
I20260810 14:53:10.535539 32145 consensus_queue.cc:237] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9651596504d149e79d18779ca8f85d6f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40607 } } peers { permanent_uuid: "819b49bace174d8bb08be94b1e699261" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 41905 } } peers { permanent_uuid: "aa7de893827e4d01a88d53667cb69818" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33375 } }
I20260810 14:53:10.536535 31631 catalog_manager.cc:5495] T d7f5fce557784c28b7836b69ff1565a4 P 9651596504d149e79d18779ca8f85d6f reported cstate change: term changed from 0 to 1, leader changed from <none> to 9651596504d149e79d18779ca8f85d6f (127.30.74.65). New cstate: current_term: 1 leader_uuid: "9651596504d149e79d18779ca8f85d6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9651596504d149e79d18779ca8f85d6f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "819b49bace174d8bb08be94b1e699261" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 41905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa7de893827e4d01a88d53667cb69818" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33375 } health_report { overall_health: UNKNOWN } } }
Aug 10 14:53:10 dist-test-slave-0b58 krb5kdc[31583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.65@KRBTEST.COM
I20260810 14:53:10.547515 32137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.539492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47356 (local address 127.30.74.65:40607)
0810 14:53:10.539576 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:10.539578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:10.539632 (+ 54us) server_negotiation.cc:407] Connection header received
0810 14:53:10.539721 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:10.539722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:10.539732 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:10.539800 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:10.540456 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.541007 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.541946 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.542145 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.542215 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.544812 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:10.544821 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:10.544822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:10.544830 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:10.546513 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:10.546836 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:10.546838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:10.546839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:10.546895 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:10.547059 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:10.547059 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:10.547060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:10.547192 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:10.547308 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:10.547356 (+ 48us) server_negotiation.cc:299] Negotiation successful
0810 14:53:10.547416 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 14:53:10 dist-test-slave-0b58 krb5kdc[31583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373590, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 14:53:10 dist-test-slave-0b58 krb5kdc[31583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373590, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.74.65@KRBTEST.COM
I20260810 14:53:10.573939 32137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.566710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47360 (local address 127.30.74.65:40607)
0810 14:53:10.566802 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:10.566804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:10.566870 (+ 66us) server_negotiation.cc:407] Connection header received
0810 14:53:10.566923 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:10.566924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:10.566934 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:10.566986 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:10.567568 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.568049 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.568880 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.569060 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.569116 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.571553 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:10.571555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:10.571556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:10.571563 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:10.572916 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:10.573239 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:10.573240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:10.573241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:10.573295 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:10.573479 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:10.573480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:10.573480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:10.573573 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:10.573645 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:10.573806 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 14:53:10.573872 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 14:53:10 dist-test-slave-0b58 krb5kdc[31583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 14:53:10 dist-test-slave-0b58 krb5kdc[31583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.65@KRBTEST.COM
I20260810 14:53:10.598466 32137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:10.591071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50800 (local address 127.30.74.65:40607)
0810 14:53:10.591144 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:10.591145 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:10.591192 (+ 47us) server_negotiation.cc:407] Connection header received
0810 14:53:10.591298 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:10.591299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:10.591325 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:10.591376 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:10.591945 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.592465 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.593175 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:10.593364 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:10.593413 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:10.595714 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:10.595716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:10.595717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:10.595725 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:10.597365 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:10.597731 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:10.597732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:10.597732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:10.597808 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:10.597995 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:10.597995 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:10.597996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:10.598095 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:10.598168 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:10.598334 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 14:53:10.598398 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 14:53:10.599128 31745 tablet_service.cc:2908] Rejecting scan request for tablet d7f5fce557784c28b7836b69ff1565a4: Uninitialized: clean time has not yet been initialized
I20260810 14:53:10.600090 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 31699
I20260810 14:53:10.607252 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 31840
W20260810 14:53:10.612773 32123 tablet.cc:2369] T d7f5fce557784c28b7836b69ff1565a4 P 819b49bace174d8bb08be94b1e699261: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:53:10.614496 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 31987
I20260810 14:53:10.621585 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 31597
2026-08-10T14:53:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2130 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:53:10 dist-test-slave-0b58 krb5kdc[32164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:53:10 dist-test-slave-0b58 krb5kdc[32164](info): set up 2 sockets
Aug 10 14:53:10 dist-test-slave-0b58 krb5kdc[32164](info): commencing operation
krb5kdc: starting...
W20260810 14:53:11.712090 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:53:11 dist-test-slave-0b58 krb5kdc[32164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:53:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:53:11.839447 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:39763
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:41113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:39763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:11.926709 32180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:11.926939 32180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:11.926962 32180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:11.928207 32180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:11.928284 32180 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 14:53:11.928300 32180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:11.928314 32180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:11.928367 32180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:11.928385 32180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:11.929994 32180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:41113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:39763
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:39763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:11.930292 32180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:11.930573 32180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:11.936193 32188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:11.936218 32185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:11.936218 32186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:11.936578 32180 server_base.cc:1034] running on GCE node
I20260810 14:53:11.936873 32180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:11.937247 32180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:11.938423 32180 hybrid_clock.cc:648] HybridClock initialized: now 1786373591938409 us; error 37 us; skew 500 ppm
Aug 10 14:53:11 dist-test-slave-0b58 krb5kdc[32164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373591, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:11.941462 32180 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:53:11.942001 32180 webserver.cc:460] Webserver started at http://127.30.74.126:37411/ using document root <none> and password file <none>
I20260810 14:53:11.942281 32180 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:11.942358 32180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:11.942512 32180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:11.943670 32180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "b46c1f63b36448d988de3d3e410dd9a9"
format_stamp: "Formatted at 2026-08-10 14:53:11 on dist-test-slave-0b58"
server_key: "3e41053126a49a61b18d91519fb65f72"
server_key_iv: "05698aa5a3ce389690df237a00361adb"
server_key_version: "encryptionkey@0"
I20260810 14:53:11.944092 32180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "b46c1f63b36448d988de3d3e410dd9a9"
format_stamp: "Formatted at 2026-08-10 14:53:11 on dist-test-slave-0b58"
server_key: "3e41053126a49a61b18d91519fb65f72"
server_key_iv: "05698aa5a3ce389690df237a00361adb"
server_key_version: "encryptionkey@0"
I20260810 14:53:11.945668 32180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:11.946558 32195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:11.946839 32180 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 14:53:11.946962 32180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "b46c1f63b36448d988de3d3e410dd9a9"
format_stamp: "Formatted at 2026-08-10 14:53:11 on dist-test-slave-0b58"
server_key: "3e41053126a49a61b18d91519fb65f72"
server_key_iv: "05698aa5a3ce389690df237a00361adb"
server_key_version: "encryptionkey@0"
I20260810 14:53:11.947050 32180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:11.963266 32180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:11.965975 32180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:11.966182 32180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:11.971513 32180 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:39763
I20260810 14:53:11.971567 32259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:39763 every 8 connection(s)
I20260810 14:53:11.971923 32180 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:11.972651 32260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:11.975158 32260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9: Bootstrap starting.
I20260810 14:53:11.975833 32260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:11.976204 32260 log.cc:824] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:11.976343 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 32180
I20260810 14:53:11.976465 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:11.976706 31017 external_mini_cluster.cc:1428] Setting key 146b2f1b0c8eb04b9ba7bb7bb59c7558
I20260810 14:53:11.977191 32260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9: No bootstrap required, opened a new log
I20260810 14:53:11.979602 32260 raft_consensus.cc:348] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46c1f63b36448d988de3d3e410dd9a9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39763 } }
I20260810 14:53:11.979828 32260 raft_consensus.cc:374] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:11.979898 32260 raft_consensus.cc:729] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b46c1f63b36448d988de3d3e410dd9a9, State: Initialized, Role: FOLLOWER
I20260810 14:53:11.980051 32260 consensus_queue.cc:260] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46c1f63b36448d988de3d3e410dd9a9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39763 } }
I20260810 14:53:11.980162 32260 raft_consensus.cc:388] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:11.980211 32260 raft_consensus.cc:482] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:11.980294 32260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:11.980911 32260 raft_consensus.cc:504] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46c1f63b36448d988de3d3e410dd9a9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39763 } }
I20260810 14:53:11.981037 32260 leader_election.cc:302] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b46c1f63b36448d988de3d3e410dd9a9; no voters:
I20260810 14:53:11.981236 32260 leader_election.cc:288] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:11.981298 32265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:11.981457 32265 raft_consensus.cc:686] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [term 1 LEADER]: Becoming Leader. State: Replica: b46c1f63b36448d988de3d3e410dd9a9, State: Running, Role: LEADER
I20260810 14:53:11.981547 32260 sys_catalog.cc:564] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:11.981617 32265 consensus_queue.cc:237] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46c1f63b36448d988de3d3e410dd9a9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39763 } }
I20260810 14:53:11.982024 32265 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b46c1f63b36448d988de3d3e410dd9a9. Latest consensus state: current_term: 1 leader_uuid: "b46c1f63b36448d988de3d3e410dd9a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46c1f63b36448d988de3d3e410dd9a9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39763 } } }
I20260810 14:53:11.983367 32265 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:11.983430 32267 sys_catalog.cc:455] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b46c1f63b36448d988de3d3e410dd9a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b46c1f63b36448d988de3d3e410dd9a9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39763 } } }
W20260810 14:53:11.983572 32280 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:53:11.983675 32280 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:53:11.983583 32267 sys_catalog.cc:458] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:11.983814 32283 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:11.984019 32283 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:11.985777 32283 catalog_manager.cc:1347] Generated new cluster ID: 5a55773310304b4e9f59c8316179df4a
I20260810 14:53:11.985834 32283 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 14:53:11 dist-test-slave-0b58 krb5kdc[32164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:11.990906 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:11.977310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58492 (local address 127.30.74.126:39763)
0810 14:53:11.977519 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:11.977528 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:11.977559 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:53:11.979591 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:11.979601 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:11.979672 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:11.979866 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:11.980648 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:11.982019 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:11.983110 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:11.983782 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:11.983907 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:11.986911 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:11.986916 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:11.986917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:11.986931 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:11.989166 (+ 2235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:11.989596 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:11.989600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:11.989600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:11.989739 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:11.989960 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:11.989961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:11.989961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:11.990301 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:11.990464 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:11.990546 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 14:53:11.990693 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":97,"threads_started":1}
I20260810 14:53:11.992915 32283 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:11.993626 32283 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:12.005740 32283 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9: Generated new TSK 0
I20260810 14:53:12.005996 32283 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:53:12.039618 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39763
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:41113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:12.125240 32287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:12.125458 32287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:12.125484 32287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:12.126672 32287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:12.126711 32287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:12.126732 32287 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:53:12.126749 32287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:53:12.128166 32287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:41113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.74.126:39763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:12.128440 32287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:12.128649 32287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:12.133774 32294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:12.133805 32293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:12.133788 32296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:12.134025 32287 server_base.cc:1034] running on GCE node
I20260810 14:53:12.134285 32287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:12.134567 32287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:12.135737 32287 hybrid_clock.cc:648] HybridClock initialized: now 1786373592135734 us; error 45 us; skew 500 ppm
Aug 10 14:53:12 dist-test-slave-0b58 krb5kdc[32164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373592, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:12.138671 32287 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:53:12.139175 32287 webserver.cc:460] Webserver started at http://127.30.74.65:42749/ using document root <none> and password file <none>
I20260810 14:53:12.139457 32287 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:12.139534 32287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:12.139698 32287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:12.140935 32287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "e99077c522de47cf9e589d9109a19ce8"
format_stamp: "Formatted at 2026-08-10 14:53:12 on dist-test-slave-0b58"
server_key: "67096c26a1fba667d1b7c2a0c02a52e3"
server_key_iv: "c7c79bbe7ec17c08ed039d19be64870b"
server_key_version: "encryptionkey@0"
I20260810 14:53:12.141480 32287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "e99077c522de47cf9e589d9109a19ce8"
format_stamp: "Formatted at 2026-08-10 14:53:12 on dist-test-slave-0b58"
server_key: "67096c26a1fba667d1b7c2a0c02a52e3"
server_key_iv: "c7c79bbe7ec17c08ed039d19be64870b"
server_key_version: "encryptionkey@0"
I20260810 14:53:12.143498 32287 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 14:53:12.144481 32303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:12.144693 32287 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:12.144816 32287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "e99077c522de47cf9e589d9109a19ce8"
format_stamp: "Formatted at 2026-08-10 14:53:12 on dist-test-slave-0b58"
server_key: "67096c26a1fba667d1b7c2a0c02a52e3"
server_key_iv: "c7c79bbe7ec17c08ed039d19be64870b"
server_key_version: "encryptionkey@0"
I20260810 14:53:12.144915 32287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:12.164001 32287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:12.166531 32287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:12.166713 32287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:12.167286 32287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:12.167347 32287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:12.167382 32287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:12.167424 32287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:12.174319 32287 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:38183
I20260810 14:53:12.174396 32422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:38183 every 8 connection(s)
I20260810 14:53:12.174734 32287 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
Aug 10 14:53:12 dist-test-slave-0b58 krb5kdc[32164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373592, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:12.175611 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 32287
I20260810 14:53:12.175722 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:53:12.175879 31017 external_mini_cluster.cc:1428] Setting key 4d23460c8bd18c4dfb9de88aea0078c9
I20260810 14:53:12.179168 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.167946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:60527 (local address 127.30.74.126:39763)
0810 14:53:12.168023 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:12.168025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:12.168238 (+ 213us) server_negotiation.cc:407] Connection header received
0810 14:53:12.168885 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:12.168886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:12.168907 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:12.168990 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:12.169898 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.170770 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.172444 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.172836 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.172924 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.175703 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:12.175704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:12.175704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:12.175719 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:12.177460 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:12.178014 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:12.178014 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:12.178015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:12.178072 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:12.178410 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:12.178411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:12.178412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:12.178572 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:12.178648 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:12.178933 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 14:53:12.179054 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 14:53:12.179200 32313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.167920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39763 (local address 127.30.74.65:60527)
0810 14:53:12.168093 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:12.168135 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:12.168734 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:12.169078 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:12.169082 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:12.169470 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:12.169774 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.169782 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.170891 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.170892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.172331 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.172335 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.172932 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.172933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.173127 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.173791 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:12.173802 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:12.175563 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:12.177553 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:12.177562 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:12.177564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:12.177932 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:12.178158 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:12.178160 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:12.178160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:12.178336 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:12.178643 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:12.178648 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:12.178798 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 14:53:12.178889 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 14:53:12.179027 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260810 14:53:12.180337 32423 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39763
I20260810 14:53:12.180446 32423 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:12.180660 32423 heartbeater.cc:511] Master 127.30.74.126:39763 requested a full tablet report, sending...
I20260810 14:53:12.181289 32212 ts_manager.cc:194] Registered new tserver with Master: e99077c522de47cf9e589d9109a19ce8 (127.30.74.65:38183)
I20260810 14:53:12.182343 32212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:60527
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
I20260810 14:53:12.187836 32430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.181326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39763 (local address 127.30.74.65:49049)
0810 14:53:12.181494 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:12.181505 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:12.181561 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:12.181706 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:12.181707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:12.181914 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:12.181982 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.181984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.182922 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.182923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.183644 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.183646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.184026 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.184028 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.184096 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.184679 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:12.184684 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:12.185026 (+ 342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:12.186781 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:12.186783 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:12.186783 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:12.187027 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:12.187271 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:12.187271 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:12.187272 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:12.187333 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:12.187576 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:12.187576 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:12.187619 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 14:53:12.187671 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 14:53:12.187763 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":77,"threads_started":1}
I20260810 14:53:12.187850 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.181365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:49049 (local address 127.30.74.126:39763)
0810 14:53:12.181552 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:12.181554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:12.181564 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:12.181642 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:12.181643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:12.181653 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:12.181703 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:12.182081 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.182826 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.183727 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.183948 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.184026 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.185139 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:12.185141 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:12.185141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:12.185150 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:12.186694 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:12.187118 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:12.187120 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:12.187120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:12.187187 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:12.187398 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:12.187399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:12.187399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:12.187516 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:12.187573 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:12.187706 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 14:53:12.187780 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:53:12.207746 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39763
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:41113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:12.295859 32432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:12.296105 32432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:12.296133 32432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:12.297839 32432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:12.297919 32432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:12.297942 32432 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:53:12.297962 32432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:53:12.299687 32432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:41113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.74.126:39763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:12.299999 32432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:12.300287 32432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:12.306000 32439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:12.306005 32438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:12.306026 32441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:12.306059 32432 server_base.cc:1034] running on GCE node
I20260810 14:53:12.306490 32432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:12.306792 32432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:12.307976 32432 hybrid_clock.cc:648] HybridClock initialized: now 1786373592307963 us; error 35 us; skew 500 ppm
Aug 10 14:53:12 dist-test-slave-0b58 krb5kdc[32164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373592, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:12.310835 32432 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:53:12.311337 32432 webserver.cc:460] Webserver started at http://127.30.74.66:45809/ using document root <none> and password file <none>
I20260810 14:53:12.311623 32432 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:12.311704 32432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:12.311849 32432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:12.313073 32432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "e176dd8920184f0584480f93ed9968b3"
format_stamp: "Formatted at 2026-08-10 14:53:12 on dist-test-slave-0b58"
server_key: "47aa9b4f5af48389d6eff2dae141416a"
server_key_iv: "d0579cf46ad3e3a2212e4b82c345d655"
server_key_version: "encryptionkey@0"
I20260810 14:53:12.313488 32432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "e176dd8920184f0584480f93ed9968b3"
format_stamp: "Formatted at 2026-08-10 14:53:12 on dist-test-slave-0b58"
server_key: "47aa9b4f5af48389d6eff2dae141416a"
server_key_iv: "d0579cf46ad3e3a2212e4b82c345d655"
server_key_version: "encryptionkey@0"
I20260810 14:53:12.314924 32432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 14:53:12.315783 32448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:12.316000 32432 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:12.316099 32432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "e176dd8920184f0584480f93ed9968b3"
format_stamp: "Formatted at 2026-08-10 14:53:12 on dist-test-slave-0b58"
server_key: "47aa9b4f5af48389d6eff2dae141416a"
server_key_iv: "d0579cf46ad3e3a2212e4b82c345d655"
server_key_version: "encryptionkey@0"
I20260810 14:53:12.316177 32432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:12.326731 32432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:12.328996 32432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:12.329167 32432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:12.329721 32432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:12.329761 32432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:12.329784 32432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:12.329797 32432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:12.337805 32432 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:37583
I20260810 14:53:12.337906 32570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:37583 every 8 connection(s)
Aug 10 14:53:12 dist-test-slave-0b58 krb5kdc[32164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373592, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:12.338281 32432 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:53:12.341737 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.330333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:41381 (local address 127.30.74.126:39763)
0810 14:53:12.330412 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:12.330415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:12.330700 (+ 285us) server_negotiation.cc:407] Connection header received
0810 14:53:12.331503 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:12.331505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:12.331516 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:12.331587 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:12.332612 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.333131 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.334896 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.335092 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.335194 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.338687 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:12.338689 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:12.338689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:12.338700 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:12.340197 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:12.340684 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:12.340685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:12.340686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:12.340756 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:12.341035 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:12.341036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:12.341036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:12.341164 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:12.341236 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:12.341546 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 14:53:12.341638 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 14:53:12.341951 32458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.330316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39763 (local address 127.30.74.66:41381)
0810 14:53:12.330538 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:12.330593 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:12.331364 (+ 771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:12.331674 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:12.331679 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:12.332117 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:12.332493 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.332502 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.333259 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.333261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.334809 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.334814 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.335206 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.335208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.335384 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.336623 (+ 1239us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:12.336635 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:12.338579 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:12.340345 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:12.340354 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:12.340356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:12.340598 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:12.340829 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:12.340830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:12.340830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:12.340957 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:12.341241 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:12.341248 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:12.341402 (+ 154us) client_negotiation.cc:769] Sending connection context
0810 14:53:12.341511 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 14:53:12.341702 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":116,"threads_started":1}
I20260810 14:53:12.342516 32571 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39763
I20260810 14:53:12.342620 32571 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:12.342824 32571 heartbeater.cc:511] Master 127.30.74.126:39763 requested a full tablet report, sending...
I20260810 14:53:12.343241 32211 ts_manager.cc:194] Registered new tserver with Master: e176dd8920184f0584480f93ed9968b3 (127.30.74.66:37583)
I20260810 14:53:12.343662 32211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:41381
I20260810 14:53:12.344005 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 32432
I20260810 14:53:12.344115 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:53:12.344370 31017 external_mini_cluster.cc:1428] Setting key 6d80b16570dea9a3fcc5d8f0cb6b6b40
I20260810 14:53:12.350862 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.343745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:47023 (local address 127.30.74.126:39763)
0810 14:53:12.344040 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:12.344042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:12.344052 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:12.344074 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:12.344075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:12.344088 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:12.344169 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:12.344701 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.345359 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.346066 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.346409 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.346487 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.347718 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:12.347719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:12.347719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:12.347726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:12.349354 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:12.349847 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:12.349849 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:12.349849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:12.349907 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:12.350162 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:12.350163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:12.350163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:12.350339 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:12.350401 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:12.350627 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 14:53:12.350762 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270}
I20260810 14:53:12.350889 32577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.343683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39763 (local address 127.30.74.66:47023)
0810 14:53:12.343831 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:12.343842 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:12.343901 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:12.344222 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:12.344223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:12.344499 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:12.344610 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.344613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.345464 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.345465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.345982 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.345984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.346509 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.346510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.346579 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.347166 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:12.347173 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:12.347617 (+ 444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:12.349453 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:12.349454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:12.349455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:12.349754 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:12.349987 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:12.349988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:12.349989 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:12.350073 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:12.350415 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:12.350417 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:12.350482 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 14:53:12.350568 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 14:53:12.350716 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":1,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:53:12.373304 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39763
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:41113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:12.461067 32580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:12.461323 32580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:12.461369 32580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:12.462610 32580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:12.462678 32580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:12.462725 32580 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:53:12.462778 32580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:53:12.464380 32580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:41113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.74.126:39763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:12.464660 32580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:12.464926 32580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:12.471717 32586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:12.471786 32580 server_base.cc:1034] running on GCE node
W20260810 14:53:12.471747 32588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:12.471747 32585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:12.472249 32580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:12.472604 32580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:12.473817 32580 hybrid_clock.cc:648] HybridClock initialized: now 1786373592473801 us; error 36 us; skew 500 ppm
Aug 10 14:53:12 dist-test-slave-0b58 krb5kdc[32164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373592, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:12.476737 32580 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:53:12.477308 32580 webserver.cc:460] Webserver started at http://127.30.74.67:36263/ using document root <none> and password file <none>
I20260810 14:53:12.477512 32580 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:12.477556 32580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:12.477665 32580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:12.478767 32580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "e1b8b6260af84d639e003abc3bdab0f1"
format_stamp: "Formatted at 2026-08-10 14:53:12 on dist-test-slave-0b58"
server_key: "24de9298090f0b6fd40c9d646c742680"
server_key_iv: "6c035c83a30b337b5bd9c71414ad3677"
server_key_version: "encryptionkey@0"
I20260810 14:53:12.479146 32580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "e1b8b6260af84d639e003abc3bdab0f1"
format_stamp: "Formatted at 2026-08-10 14:53:12 on dist-test-slave-0b58"
server_key: "24de9298090f0b6fd40c9d646c742680"
server_key_iv: "6c035c83a30b337b5bd9c71414ad3677"
server_key_version: "encryptionkey@0"
I20260810 14:53:12.480887 32580 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260810 14:53:12.481956 32596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:12.482182 32580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:12.482297 32580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "e1b8b6260af84d639e003abc3bdab0f1"
format_stamp: "Formatted at 2026-08-10 14:53:12 on dist-test-slave-0b58"
server_key: "24de9298090f0b6fd40c9d646c742680"
server_key_iv: "6c035c83a30b337b5bd9c71414ad3677"
server_key_version: "encryptionkey@0"
I20260810 14:53:12.482386 32580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:12.495972 32580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:12.498692 32580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:12.498867 32580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:12.499523 32580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:12.499574 32580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:12.499598 32580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:12.499611 32580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:12.506872 32580 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:42785
I20260810 14:53:12.506953 32721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:42785 every 8 connection(s)
Aug 10 14:53:12 dist-test-slave-0b58 krb5kdc[32164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373592, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:12.507269 32580 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:53:12.509614 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 32580
I20260810 14:53:12.509744 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:53:12.509917 31017 external_mini_cluster.cc:1428] Setting key 0ef4b8b223252145fe26b74e465e0caa
I20260810 14:53:12.511001 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.500084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:47015 (local address 127.30.74.126:39763)
0810 14:53:12.500174 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:12.500176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:12.500418 (+ 242us) server_negotiation.cc:407] Connection header received
0810 14:53:12.500929 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:12.500930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:12.500945 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:12.501026 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:12.501982 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.502866 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.504559 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.504878 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.504969 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.507832 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:12.507833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:12.507833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:12.507841 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:12.509219 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:12.509787 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:12.509794 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:12.509798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:12.509876 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:12.510222 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:12.510224 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:12.510224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:12.510389 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:12.510458 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:12.510786 (+ 328us) server_negotiation.cc:299] Negotiation successful
0810 14:53:12.510910 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 14:53:12.511210 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.500044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39763 (local address 127.30.74.67:47015)
0810 14:53:12.500278 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:12.500324 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:12.500829 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:12.501101 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:12.501105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:12.501519 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:12.501846 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.501854 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.502984 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.502985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.504439 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.504443 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.504974 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.504975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.505084 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.505827 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:12.505838 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:12.507751 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:12.509319 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:12.509327 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:12.509329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:12.509612 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:12.509963 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:12.509964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:12.509965 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:12.510115 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:12.510461 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:12.510468 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:12.510640 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 14:53:12.510760 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 14:53:12.510949 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":112,"threads_started":1}
I20260810 14:53:12.511674 32722 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39763
I20260810 14:53:12.511787 32722 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:12.511982 32722 heartbeater.cc:511] Master 127.30.74.126:39763 requested a full tablet report, sending...
I20260810 14:53:12.512452 32212 ts_manager.cc:194] Registered new tserver with Master: e1b8b6260af84d639e003abc3bdab0f1 (127.30.74.67:42785)
I20260810 14:53:12.512948 32212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:47015
I20260810 14:53:12.518404 32728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.512794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39763 (local address 127.30.74.67:41537)
0810 14:53:12.512953 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:12.512965 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:12.513031 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:12.513216 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:12.513217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:12.513384 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:12.513433 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.513435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.514238 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.514239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.514760 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.514761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.515115 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.515116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.515170 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.515583 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:12.515587 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:12.515874 (+ 287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:12.517503 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:12.517504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:12.517504 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:12.517693 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:12.517908 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:12.517908 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:12.517909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:12.517964 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:12.518146 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:12.518147 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:12.518188 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 14:53:12.518239 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 14:53:12.518319 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":102,"threads_started":1}
I20260810 14:53:12.518455 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.512851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:41537 (local address 127.30.74.126:39763)
0810 14:53:12.512896 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:12.512899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:12.513021 (+ 122us) server_negotiation.cc:407] Connection header received
0810 14:53:12.513118 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:12.513119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:12.513130 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:12.513198 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:12.513503 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.514142 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.514853 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.515040 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.515105 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.515954 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:12.515955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:12.515955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:12.515962 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:12.517407 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:12.517783 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:12.517784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:12.517784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:12.517842 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:12.518004 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:12.518004 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:12.518004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:12.518098 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:12.518143 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:12.518255 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 14:53:12.518371 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:53:12.521811 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:53:12 dist-test-slave-0b58 krb5kdc[32164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373592, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 14:53:12 dist-test-slave-0b58 krb5kdc[32164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373592, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:12.547967 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.540068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58508 (local address 127.30.74.126:39763)
0810 14:53:12.540165 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:12.540167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:12.540180 (+ 13us) server_negotiation.cc:407] Connection header received
0810 14:53:12.540232 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:12.540233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:12.540244 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:12.540355 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:12.540956 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.541467 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.542239 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.542433 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.542547 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.545201 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:12.545202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:12.545203 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:12.545211 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:12.546735 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:12.547101 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:12.547102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:12.547102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:12.547159 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:12.547389 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:12.547390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:12.547390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:12.547501 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:12.547578 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:12.547790 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 14:53:12.547860 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 14:53:12.549754 32212 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:53:12.551525 32212 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:12.559067 32741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.554496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:42785 (local address 127.0.0.1:54344)
0810 14:53:12.554659 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:12.554707 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:12.554778 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:12.555235 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:12.555240 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:12.555242 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:12.555328 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.555330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.556796 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.556797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.557875 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.557878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.558656 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.558658 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.558790 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.558809 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 14:53:12.558882 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 14:53:12.558947 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":56,"threads_started":1}
I20260810 14:53:12.559104 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.554796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53978 (local address 127.30.74.65:38183)
0810 14:53:12.555003 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:12.555006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:12.555287 (+ 281us) server_negotiation.cc:407] Connection header received
0810 14:53:12.555332 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:12.555335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:12.555351 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:12.555422 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:12.555586 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.556415 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.557980 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.558717 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.558798 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.558853 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:12.558948 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 14:53:12.558983 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":101,"threads_started":1}
I20260810 14:53:12.559259 32744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.554593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54344 (local address 127.30.74.67:42785)
0810 14:53:12.554928 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:12.554934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:12.555010 (+ 76us) server_negotiation.cc:407] Connection header received
0810 14:53:12.555057 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:12.555059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:12.555077 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:12.555413 (+ 336us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:12.555475 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.556706 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.557948 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.558521 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.558631 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.558699 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:12.559111 (+ 412us) server_negotiation.cc:299] Negotiation successful
0810 14:53:12.559159 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":143,"threads_started":1}
I20260810 14:53:12.559067 32742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.554511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:38183 (local address 127.0.0.1:53978)
0810 14:53:12.555022 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:12.555032 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:12.555092 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:12.555493 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:12.555494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:12.555496 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:12.555529 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.555530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.556502 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.556503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.557870 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.557874 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.558806 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.558807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.558869 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.558872 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:53:12.558932 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 14:53:12.558962 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":93,"threads_started":1}
I20260810 14:53:12.560333 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.554510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:37583 (local address 127.0.0.1:39296)
0810 14:53:12.555167 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:12.555175 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:12.555220 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:12.555390 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:12.555391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:12.555393 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:12.555453 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.555454 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.557972 (+ 2518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.557973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.559038 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.559042 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.560039 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.560040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.560107 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.560112 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:53:12.560182 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 14:53:12.560233 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":3,"thread_start_us":97,"threads_started":1}
I20260810 14:53:12.560156 32650 tablet_service.cc:1467] Processing CreateTablet for tablet 9927ec8466894883b0ac9490d675189e (DEFAULT_TABLE table=test-table [id=9c724f2fa82e42358d0c69b953749d41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:12.560381 32745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.554767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39296 (local address 127.30.74.66:37583)
0810 14:53:12.554964 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:12.554969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:12.555227 (+ 258us) server_negotiation.cc:407] Connection header received
0810 14:53:12.555279 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:12.555281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:12.555297 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:12.555380 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:12.555514 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.556695 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.559144 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.559952 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.560028 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.560086 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:12.560205 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 14:53:12.560244 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":89,"threads_started":1}
I20260810 14:53:12.560431 32354 tablet_service.cc:1467] Processing CreateTablet for tablet 9927ec8466894883b0ac9490d675189e (DEFAULT_TABLE table=test-table [id=9c724f2fa82e42358d0c69b953749d41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:12.560622 32650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9927ec8466894883b0ac9490d675189e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:12.560834 32354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9927ec8466894883b0ac9490d675189e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:12.561290 32501 tablet_service.cc:1467] Processing CreateTablet for tablet 9927ec8466894883b0ac9490d675189e (DEFAULT_TABLE table=test-table [id=9c724f2fa82e42358d0c69b953749d41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:12.561627 32501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9927ec8466894883b0ac9490d675189e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:12.563519 32747 tablet_bootstrap.cc:492] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1: Bootstrap starting.
I20260810 14:53:12.564713 32747 tablet_bootstrap.cc:654] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:12.565066 32747 log.cc:824] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:12.565315 32750 tablet_bootstrap.cc:492] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3: Bootstrap starting.
I20260810 14:53:12.565564 32749 tablet_bootstrap.cc:492] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8: Bootstrap starting.
I20260810 14:53:12.565909 32747 tablet_bootstrap.cc:492] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1: No bootstrap required, opened a new log
I20260810 14:53:12.565997 32747 ts_tablet_manager.cc:1397] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 14:53:12.566011 32750 tablet_bootstrap.cc:654] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:12.566286 32749 tablet_bootstrap.cc:654] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:12.566385 32750 log.cc:824] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:12.566644 32749 log.cc:824] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:12.567262 32750 tablet_bootstrap.cc:492] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3: No bootstrap required, opened a new log
I20260810 14:53:12.567322 32750 ts_tablet_manager.cc:1397] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:53:12.567523 32749 tablet_bootstrap.cc:492] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8: No bootstrap required, opened a new log
I20260810 14:53:12.567651 32749 ts_tablet_manager.cc:1397] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:53:12.568305 32747 raft_consensus.cc:348] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } }
I20260810 14:53:12.568499 32747 raft_consensus.cc:374] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:12.568539 32747 raft_consensus.cc:729] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1b8b6260af84d639e003abc3bdab0f1, State: Initialized, Role: FOLLOWER
I20260810 14:53:12.568687 32747 consensus_queue.cc:260] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } }
I20260810 14:53:12.568995 32747 ts_tablet_manager.cc:1428] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 14:53:12.569062 32722 heartbeater.cc:503] Master 127.30.74.126:39763 was elected leader, sending a full tablet report...
I20260810 14:53:12.569202 32750 raft_consensus.cc:348] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } }
I20260810 14:53:12.569361 32750 raft_consensus.cc:374] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:12.569412 32750 raft_consensus.cc:729] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e176dd8920184f0584480f93ed9968b3, State: Initialized, Role: FOLLOWER
I20260810 14:53:12.569540 32750 consensus_queue.cc:260] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } }
I20260810 14:53:12.569846 32750 ts_tablet_manager.cc:1428] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 14:53:12.569926 32571 heartbeater.cc:503] Master 127.30.74.126:39763 was elected leader, sending a full tablet report...
I20260810 14:53:12.570417 32749 raft_consensus.cc:348] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } }
I20260810 14:53:12.570572 32749 raft_consensus.cc:374] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:12.570611 32749 raft_consensus.cc:729] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e99077c522de47cf9e589d9109a19ce8, State: Initialized, Role: FOLLOWER
I20260810 14:53:12.570729 32749 consensus_queue.cc:260] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } }
I20260810 14:53:12.570986 32749 ts_tablet_manager.cc:1428] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 14:53:12.571144 32423 heartbeater.cc:503] Master 127.30.74.126:39763 was elected leader, sending a full tablet report...
W20260810 14:53:12.589018 32572 tablet.cc:2369] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:53:12.675555 32424 tablet.cc:2369] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:53:12.757872 32723 tablet.cc:2369] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:53:12.836860 32754 raft_consensus.cc:482] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:12.837006 32754 raft_consensus.cc:504] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } }
I20260810 14:53:12.837477 32754 leader_election.cc:288] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e176dd8920184f0584480f93ed9968b3 (127.30.74.66:37583), e99077c522de47cf9e589d9109a19ce8 (127.30.74.65:38183)
I20260810 14:53:12.840955 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.837759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:37583 (local address 127.30.74.67:44185)
0810 14:53:12.837841 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:12.837852 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:12.837905 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:12.838109 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:12.838110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:12.838111 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:12.838225 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.838227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.838953 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.838954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.840076 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.840079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.840671 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.840672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.840769 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.840773 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:12.840839 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 14:53:12.840879 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 14:53:12.840955 32745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.837829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:44185 (local address 127.30.74.66:37583)
0810 14:53:12.837893 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:12.837895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:12.837911 (+ 16us) server_negotiation.cc:407] Connection header received
0810 14:53:12.837982 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:12.837983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:12.837991 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:12.838098 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:12.838308 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.838866 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.840178 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.840567 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.840662 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.840671 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:12.840846 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 14:53:12.840884 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 14:53:12.841351 32522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9927ec8466894883b0ac9490d675189e" candidate_uuid: "e1b8b6260af84d639e003abc3bdab0f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e176dd8920184f0584480f93ed9968b3" is_pre_election: true
I20260810 14:53:12.841446 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.837850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:34711 (local address 127.30.74.65:38183)
0810 14:53:12.837903 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:12.837906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:12.838157 (+ 251us) server_negotiation.cc:407] Connection header received
0810 14:53:12.838245 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:12.838246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:12.838255 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:12.838300 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:12.838468 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.839011 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.840173 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.840621 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.840717 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.840726 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:12.841360 (+ 634us) server_negotiation.cc:299] Negotiation successful
0810 14:53:12.841395 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 14:53:12.841475 32757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.837759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:38183 (local address 127.30.74.67:34711)
0810 14:53:12.838096 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:12.838109 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:12.838173 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:12.838312 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:12.838313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:12.838314 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:12.838388 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.838390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.839098 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.839099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.840078 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:12.840081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.840719 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.840720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:12.841235 (+ 515us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.841239 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:12.841333 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 14:53:12.841381 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":125,"thread_start_us":129,"threads_started":1}
I20260810 14:53:12.841535 32522 raft_consensus.cc:2454] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1b8b6260af84d639e003abc3bdab0f1 in term 0.
I20260810 14:53:12.841825 32598 leader_election.cc:302] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e176dd8920184f0584480f93ed9968b3, e1b8b6260af84d639e003abc3bdab0f1; no voters:
I20260810 14:53:12.841915 32374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9927ec8466894883b0ac9490d675189e" candidate_uuid: "e1b8b6260af84d639e003abc3bdab0f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e99077c522de47cf9e589d9109a19ce8" is_pre_election: true
I20260810 14:53:12.841976 32754 raft_consensus.cc:2781] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:12.842024 32754 raft_consensus.cc:482] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:12.842057 32754 raft_consensus.cc:3037] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:12.842073 32374 raft_consensus.cc:2454] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1b8b6260af84d639e003abc3bdab0f1 in term 0.
I20260810 14:53:12.842941 32754 raft_consensus.cc:504] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } }
I20260810 14:53:12.843091 32754 leader_election.cc:288] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [CANDIDATE]: Term 1 election: Requested vote from peers e176dd8920184f0584480f93ed9968b3 (127.30.74.66:37583), e99077c522de47cf9e589d9109a19ce8 (127.30.74.65:38183)
I20260810 14:53:12.843256 32522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9927ec8466894883b0ac9490d675189e" candidate_uuid: "e1b8b6260af84d639e003abc3bdab0f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e176dd8920184f0584480f93ed9968b3"
I20260810 14:53:12.843346 32522 raft_consensus.cc:3037] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:12.843327 32374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9927ec8466894883b0ac9490d675189e" candidate_uuid: "e1b8b6260af84d639e003abc3bdab0f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e99077c522de47cf9e589d9109a19ce8"
I20260810 14:53:12.843446 32374 raft_consensus.cc:3037] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:12.844111 32522 raft_consensus.cc:2454] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1b8b6260af84d639e003abc3bdab0f1 in term 1.
I20260810 14:53:12.844317 32374 raft_consensus.cc:2454] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1b8b6260af84d639e003abc3bdab0f1 in term 1.
I20260810 14:53:12.844319 32598 leader_election.cc:302] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e176dd8920184f0584480f93ed9968b3, e1b8b6260af84d639e003abc3bdab0f1; no voters:
I20260810 14:53:12.844439 32754 raft_consensus.cc:2781] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:12.844587 32754 raft_consensus.cc:686] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [term 1 LEADER]: Becoming Leader. State: Replica: e1b8b6260af84d639e003abc3bdab0f1, State: Running, Role: LEADER
I20260810 14:53:12.844674 32754 consensus_queue.cc:237] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } }
I20260810 14:53:12.845510 32211 catalog_manager.cc:5495] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1b8b6260af84d639e003abc3bdab0f1 (127.30.74.67). New cstate: current_term: 1 leader_uuid: "e1b8b6260af84d639e003abc3bdab0f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } health_report { overall_health: UNKNOWN } } }
I20260810 14:53:12.915493 32744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:12.913029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54350 (local address 127.30.74.67:42785)
0810 14:53:12.913101 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:12.913103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:12.913112 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:12.913203 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:12.913204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:12.913222 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:12.913288 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:12.913441 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.913918 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.914684 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:12.914887 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:12.914940 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:12.915128 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:12.915221 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:12.915285 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:12.915342 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 14:53:12.915360 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 14:53:12.917410 32374 raft_consensus.cc:1261] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8 [term 1 FOLLOWER]: Refusing update from remote peer e1b8b6260af84d639e003abc3bdab0f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:12.917418 32522 raft_consensus.cc:1261] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3 [term 1 FOLLOWER]: Refusing update from remote peer e1b8b6260af84d639e003abc3bdab0f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:12.917829 32758 consensus_queue.cc:1035] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:12.917968 32754 consensus_queue.cc:1035] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:12.920993 32763 mvcc.cc:205] Tried to move back new op lower bound from 7316986236586909696 to 7316986236292288512. Current Snapshot: MvccSnapshot[applied={T|T < 7316986236586909696}]
I20260810 14:53:12.924703 32211 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:12.924875 32211 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:12.925887 32211 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9: Sending DeleteTablet for 3 replicas of tablet 9927ec8466894883b0ac9490d675189e
I20260810 14:53:12.926251 32501 tablet_service.cc:1514] Processing DeleteTablet for tablet 9927ec8466894883b0ac9490d675189e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:12 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:39296
I20260810 14:53:12.926291 32650 tablet_service.cc:1514] Processing DeleteTablet for tablet 9927ec8466894883b0ac9490d675189e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:12 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:54344
I20260810 14:53:12.926524 303 tablet_replica.cc:330] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1: stopping tablet replica
I20260810 14:53:12.926512 32354 tablet_service.cc:1514] Processing DeleteTablet for tablet 9927ec8466894883b0ac9490d675189e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:12 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:53978
I20260810 14:53:12.926620 303 raft_consensus.cc:2229] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:53:12.926604 32211 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 14:53:12.926734 304 tablet_replica.cc:330] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8: stopping tablet replica
W20260810 14:53:12.926757 32211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:12.926764 303 raft_consensus.cc:2258] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:12.926824 304 raft_consensus.cc:2229] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:12.926930 304 raft_consensus.cc:2258] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:12.927218 303 ts_tablet_manager.cc:1905] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:12.927371 304 ts_tablet_manager.cc:1905] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:12.928884 32354 tablet_service.cc:1467] Processing CreateTablet for tablet 1866930e9fa647318c211b0006d50e04 (DEFAULT_TABLE table=test-table [id=819152da626c42569a7c4fbdd7af2109]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:12.929024 304 ts_tablet_manager.cc:1918] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:12.929024 303 ts_tablet_manager.cc:1918] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:12.929059 32354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1866930e9fa647318c211b0006d50e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:12.929100 304 log.cc:1196] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/wal/wals/9927ec8466894883b0ac9490d675189e
I20260810 14:53:12.929100 303 log.cc:1196] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/9927ec8466894883b0ac9490d675189e
I20260810 14:53:12.929378 303 ts_tablet_manager.cc:1939] T 9927ec8466894883b0ac9490d675189e P e1b8b6260af84d639e003abc3bdab0f1: Deleting consensus metadata
I20260810 14:53:12.929378 304 ts_tablet_manager.cc:1939] T 9927ec8466894883b0ac9490d675189e P e99077c522de47cf9e589d9109a19ce8: Deleting consensus metadata
I20260810 14:53:12.929994 32199 catalog_manager.cc:4841] TS e99077c522de47cf9e589d9109a19ce8 (127.30.74.65:38183): tablet 9927ec8466894883b0ac9490d675189e (table test-table [id=9c724f2fa82e42358d0c69b953749d41]) successfully deleted
I20260810 14:53:12.930519 32749 tablet_bootstrap.cc:492] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8: Bootstrap starting.
I20260810 14:53:12.930542 32650 tablet_service.cc:1467] Processing CreateTablet for tablet 1866930e9fa647318c211b0006d50e04 (DEFAULT_TABLE table=test-table [id=819152da626c42569a7c4fbdd7af2109]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:12.930660 32650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1866930e9fa647318c211b0006d50e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:12.930981 32749 tablet_bootstrap.cc:654] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:12.931564 32749 tablet_bootstrap.cc:492] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8: No bootstrap required, opened a new log
I20260810 14:53:12.931597 32749 ts_tablet_manager.cc:1397] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:12.931713 32749 raft_consensus.cc:348] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:12.931771 32749 raft_consensus.cc:374] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:12.931784 32749 raft_consensus.cc:729] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e99077c522de47cf9e589d9109a19ce8, State: Initialized, Role: FOLLOWER
I20260810 14:53:12.931826 32749 consensus_queue.cc:260] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:12.931921 32749 ts_tablet_manager.cc:1428] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:12.931937 32747 tablet_bootstrap.cc:492] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1: Bootstrap starting.
I20260810 14:53:12.932379 32747 tablet_bootstrap.cc:654] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:12.932428 32198 catalog_manager.cc:4841] TS e1b8b6260af84d639e003abc3bdab0f1 (127.30.74.67:42785): tablet 9927ec8466894883b0ac9490d675189e (table test-table [id=9c724f2fa82e42358d0c69b953749d41]) successfully deleted
I20260810 14:53:12.932708 32501 tablet_service.cc:1467] Processing CreateTablet for tablet 1866930e9fa647318c211b0006d50e04 (DEFAULT_TABLE table=test-table [id=819152da626c42569a7c4fbdd7af2109]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:12.932850 32501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1866930e9fa647318c211b0006d50e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:12.933030 302 tablet_replica.cc:330] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3: stopping tablet replica
I20260810 14:53:12.933183 302 raft_consensus.cc:2229] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:12.933329 302 raft_consensus.cc:2258] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:12.934122 32747 tablet_bootstrap.cc:492] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1: No bootstrap required, opened a new log
I20260810 14:53:12.934180 32747 ts_tablet_manager.cc:1397] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 14:53:12.934350 32747 raft_consensus.cc:348] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:12.934405 32750 tablet_bootstrap.cc:492] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3: Bootstrap starting.
I20260810 14:53:12.934428 32747 raft_consensus.cc:374] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:12.934454 32747 raft_consensus.cc:729] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1b8b6260af84d639e003abc3bdab0f1, State: Initialized, Role: FOLLOWER
I20260810 14:53:12.934501 32747 consensus_queue.cc:260] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:12.934602 32747 ts_tablet_manager.cc:1428] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:12.934824 32750 tablet_bootstrap.cc:654] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:12.934929 302 ts_tablet_manager.cc:1905] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:12.935639 32750 tablet_bootstrap.cc:492] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3: No bootstrap required, opened a new log
I20260810 14:53:12.935693 32750 ts_tablet_manager.cc:1397] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:12.935840 32750 raft_consensus.cc:348] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:12.935912 32750 raft_consensus.cc:374] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:12.935937 32750 raft_consensus.cc:729] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e176dd8920184f0584480f93ed9968b3, State: Initialized, Role: FOLLOWER
I20260810 14:53:12.935979 32750 consensus_queue.cc:260] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:12.936287 32750 ts_tablet_manager.cc:1428] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:12.936689 302 ts_tablet_manager.cc:1918] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:12.936762 302 log.cc:1196] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/9927ec8466894883b0ac9490d675189e
I20260810 14:53:12.937050 302 ts_tablet_manager.cc:1939] T 9927ec8466894883b0ac9490d675189e P e176dd8920184f0584480f93ed9968b3: Deleting consensus metadata
I20260810 14:53:12.937491 32197 catalog_manager.cc:4841] TS e176dd8920184f0584480f93ed9968b3 (127.30.74.66:37583): tablet 9927ec8466894883b0ac9490d675189e (table test-table [id=9c724f2fa82e42358d0c69b953749d41]) successfully deleted
I20260810 14:53:13.047518 32756 raft_consensus.cc:482] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:13.047624 32756 raft_consensus.cc:504] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:13.047910 32756 leader_election.cc:288] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e176dd8920184f0584480f93ed9968b3 (127.30.74.66:37583), e1b8b6260af84d639e003abc3bdab0f1 (127.30.74.67:42785)
I20260810 14:53:13.051453 32744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:13.048371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:51471 (local address 127.30.74.67:42785)
0810 14:53:13.048422 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:13.048424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:13.048436 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:53:13.048512 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:13.048513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:13.048524 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:13.048577 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:13.048792 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.049328 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.050465 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.051015 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.051072 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:13.051079 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:13.051332 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 14:53:13.051373 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 14:53:13.051455 32745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:13.048217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:60693 (local address 127.30.74.66:37583)
0810 14:53:13.048282 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:13.048284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:13.048432 (+ 148us) server_negotiation.cc:407] Connection header received
0810 14:53:13.048529 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:13.048530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:13.048546 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:13.048621 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:13.048814 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.049287 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.050419 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.050966 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.051066 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:13.051076 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:13.051338 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 14:53:13.051382 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:53:13.051491 309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:13.048117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:42785 (local address 127.30.74.65:51471)
0810 14:53:13.048356 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:13.048369 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:13.048441 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:13.048607 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:13.048608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:13.048610 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:13.048697 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:13.048699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.049408 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.049409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:13.050363 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:13.050366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.051115 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.051116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:13.051194 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:13.051201 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 14:53:13.051317 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 14:53:13.051366 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":21,"thread_start_us":80,"threads_started":1}
I20260810 14:53:13.051496 308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:13.048178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:37583 (local address 127.30.74.65:60693)
0810 14:53:13.048358 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:13.048370 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:13.048448 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:13.048642 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:13.048643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:13.048645 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:13.048713 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:13.048715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.049377 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.049378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:13.050324 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:13.050327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.051133 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.051134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:13.051217 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:13.051222 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:53:13.051318 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 14:53:13.051368 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":27,"thread_start_us":109,"threads_started":1}
I20260810 14:53:13.051754 32672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1866930e9fa647318c211b0006d50e04" candidate_uuid: "e99077c522de47cf9e589d9109a19ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1b8b6260af84d639e003abc3bdab0f1" is_pre_election: true
I20260810 14:53:13.051892 32672 raft_consensus.cc:2454] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e99077c522de47cf9e589d9109a19ce8 in term 0.
I20260810 14:53:13.051851 32522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1866930e9fa647318c211b0006d50e04" candidate_uuid: "e99077c522de47cf9e589d9109a19ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e176dd8920184f0584480f93ed9968b3" is_pre_election: true
I20260810 14:53:13.051944 32522 raft_consensus.cc:2454] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e99077c522de47cf9e589d9109a19ce8 in term 0.
I20260810 14:53:13.052098 32306 leader_election.cc:302] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e1b8b6260af84d639e003abc3bdab0f1, e99077c522de47cf9e589d9109a19ce8; no voters:
I20260810 14:53:13.052286 32756 raft_consensus.cc:2781] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:13.052332 32756 raft_consensus.cc:482] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:13.052352 32756 raft_consensus.cc:3037] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:13.053077 32756 raft_consensus.cc:504] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:13.053226 32756 leader_election.cc:288] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [CANDIDATE]: Term 1 election: Requested vote from peers e176dd8920184f0584480f93ed9968b3 (127.30.74.66:37583), e1b8b6260af84d639e003abc3bdab0f1 (127.30.74.67:42785)
I20260810 14:53:13.053399 32672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1866930e9fa647318c211b0006d50e04" candidate_uuid: "e99077c522de47cf9e589d9109a19ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1b8b6260af84d639e003abc3bdab0f1"
I20260810 14:53:13.053496 32672 raft_consensus.cc:3037] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:13.053448 32522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1866930e9fa647318c211b0006d50e04" candidate_uuid: "e99077c522de47cf9e589d9109a19ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e176dd8920184f0584480f93ed9968b3"
I20260810 14:53:13.053571 32522 raft_consensus.cc:3037] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:13.054178 32522 raft_consensus.cc:2454] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e99077c522de47cf9e589d9109a19ce8 in term 1.
I20260810 14:53:13.054219 32672 raft_consensus.cc:2454] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e99077c522de47cf9e589d9109a19ce8 in term 1.
I20260810 14:53:13.054414 32306 leader_election.cc:302] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: e176dd8920184f0584480f93ed9968b3, e1b8b6260af84d639e003abc3bdab0f1, e99077c522de47cf9e589d9109a19ce8; no voters:
I20260810 14:53:13.054533 32756 raft_consensus.cc:2781] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:13.054675 32756 raft_consensus.cc:686] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [term 1 LEADER]: Becoming Leader. State: Replica: e99077c522de47cf9e589d9109a19ce8, State: Running, Role: LEADER
I20260810 14:53:13.054733 32756 consensus_queue.cc:237] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:13.055508 32211 catalog_manager.cc:5495] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e99077c522de47cf9e589d9109a19ce8 (127.30.74.65). New cstate: current_term: 1 leader_uuid: "e99077c522de47cf9e589d9109a19ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } health_report { overall_health: UNKNOWN } } }
Aug 10 14:53:13 dist-test-slave-0b58 krb5kdc[32164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373591, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:13.104383 318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:13.097094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58510 (local address 127.30.74.126:39763)
0810 14:53:13.097272 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:13.097274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:13.097301 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:53:13.097379 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:13.097380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:13.097390 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:13.097457 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:13.097911 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.098472 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.099161 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.099353 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.099405 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:13.101611 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:13.101613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:13.101613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:13.101620 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:13.103110 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:13.103508 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:13.103509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:13.103510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:13.103577 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:13.103813 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:13.103814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:13.103814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:13.103924 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:13.103976 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:13.104124 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 14:53:13.104230 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":105,"threads_started":1}
I20260810 14:53:13.104580 317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:13.096999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39763 (local address 127.0.0.1:58510)
0810 14:53:13.097233 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:13.097245 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:13.097320 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:13.097476 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:13.097476 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:13.097796 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:13.097845 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:13.097847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.098553 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.098554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:13.099087 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:13.099089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.099423 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.099424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:13.099468 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:13.099994 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:13.099999 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:13.101520 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:13.103197 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:13.103204 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:13.103206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:13.103420 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:13.103674 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:13.103675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:13.103675 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:13.103744 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:13.103985 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:13.103993 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:13.104037 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 14:53:13.104090 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 14:53:13.104435 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":106,"threads_started":1}
I20260810 14:53:13.107059 32211 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:58510:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 14:53:13.107271 32211 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:13.108867 32650 tablet_service.cc:1467] Processing CreateTablet for tablet 228541ff58074c978698183453e16041 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9877dbcab5514d00ad66982ea4ba21f9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:53:13.108862 32354 tablet_service.cc:1467] Processing CreateTablet for tablet 228541ff58074c978698183453e16041 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9877dbcab5514d00ad66982ea4ba21f9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:53:13.108850 32501 tablet_service.cc:1467] Processing CreateTablet for tablet 228541ff58074c978698183453e16041 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9877dbcab5514d00ad66982ea4ba21f9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:53:13.109023 32501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228541ff58074c978698183453e16041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:13.109027 32354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228541ff58074c978698183453e16041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:13.109043 32650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 228541ff58074c978698183453e16041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:13.111084 32747 tablet_bootstrap.cc:492] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1: Bootstrap starting.
I20260810 14:53:13.111114 32749 tablet_bootstrap.cc:492] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8: Bootstrap starting.
I20260810 14:53:13.111479 32747 tablet_bootstrap.cc:654] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:13.111541 32749 tablet_bootstrap.cc:654] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:13.112087 32747 tablet_bootstrap.cc:492] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1: No bootstrap required, opened a new log
I20260810 14:53:13.112140 32747 ts_tablet_manager.cc:1397] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:13.112337 32749 tablet_bootstrap.cc:492] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8: No bootstrap required, opened a new log
I20260810 14:53:13.112301 32747 raft_consensus.cc:348] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:13.112377 32747 raft_consensus.cc:374] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:13.112377 32749 ts_tablet_manager.cc:1397] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:13.112399 32747 raft_consensus.cc:729] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1b8b6260af84d639e003abc3bdab0f1, State: Initialized, Role: FOLLOWER
I20260810 14:53:13.112452 32747 consensus_queue.cc:260] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:13.112541 32747 ts_tablet_manager.cc:1428] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:13.112516 32749 raft_consensus.cc:348] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:13.112576 32749 raft_consensus.cc:374] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:13.112602 32749 raft_consensus.cc:729] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e99077c522de47cf9e589d9109a19ce8, State: Initialized, Role: FOLLOWER
I20260810 14:53:13.112552 32754 tablet_replica.cc:439] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } }
I20260810 14:53:13.112634 32754 tablet_replica.cc:442] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:13.112646 32749 consensus_queue.cc:260] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:13.112735 32749 ts_tablet_manager.cc:1428] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:13.112779 32756 tablet_replica.cc:439] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } }
I20260810 14:53:13.112838 32756 tablet_replica.cc:442] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:13.113611 32750 tablet_bootstrap.cc:492] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3: Bootstrap starting.
I20260810 14:53:13.114046 32750 tablet_bootstrap.cc:654] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:13.114655 32750 tablet_bootstrap.cc:492] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3: No bootstrap required, opened a new log
I20260810 14:53:13.114717 32750 ts_tablet_manager.cc:1397] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:13.114871 32750 raft_consensus.cc:348] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:13.114933 32750 raft_consensus.cc:374] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:13.114946 32750 raft_consensus.cc:729] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e176dd8920184f0584480f93ed9968b3, State: Initialized, Role: FOLLOWER
I20260810 14:53:13.114980 32750 consensus_queue.cc:260] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:13.115067 32750 ts_tablet_manager.cc:1428] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:13.115206 32755 tablet_replica.cc:439] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } }
I20260810 14:53:13.115270 32755 tablet_replica.cc:442] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:13.162740 32756 raft_consensus.cc:482] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:13.162843 32756 raft_consensus.cc:504] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:13.163029 32756 leader_election.cc:288] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e176dd8920184f0584480f93ed9968b3 (127.30.74.66:37583), e1b8b6260af84d639e003abc3bdab0f1 (127.30.74.67:42785)
I20260810 14:53:13.163332 32672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "228541ff58074c978698183453e16041" candidate_uuid: "e99077c522de47cf9e589d9109a19ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1b8b6260af84d639e003abc3bdab0f1" is_pre_election: true
I20260810 14:53:13.163332 32522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "228541ff58074c978698183453e16041" candidate_uuid: "e99077c522de47cf9e589d9109a19ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e176dd8920184f0584480f93ed9968b3" is_pre_election: true
I20260810 14:53:13.163450 32522 raft_consensus.cc:2454] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e99077c522de47cf9e589d9109a19ce8 in term 0.
I20260810 14:53:13.163450 32672 raft_consensus.cc:2454] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e99077c522de47cf9e589d9109a19ce8 in term 0.
I20260810 14:53:13.163627 32306 leader_election.cc:302] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e1b8b6260af84d639e003abc3bdab0f1, e99077c522de47cf9e589d9109a19ce8; no voters:
I20260810 14:53:13.163753 32756 raft_consensus.cc:2781] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:13.163811 32756 raft_consensus.cc:482] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:13.163842 32756 raft_consensus.cc:3037] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:13.164610 32756 raft_consensus.cc:504] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:13.164788 32756 leader_election.cc:288] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [CANDIDATE]: Term 1 election: Requested vote from peers e176dd8920184f0584480f93ed9968b3 (127.30.74.66:37583), e1b8b6260af84d639e003abc3bdab0f1 (127.30.74.67:42785)
I20260810 14:53:13.164948 32522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "228541ff58074c978698183453e16041" candidate_uuid: "e99077c522de47cf9e589d9109a19ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e176dd8920184f0584480f93ed9968b3"
I20260810 14:53:13.164965 32672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "228541ff58074c978698183453e16041" candidate_uuid: "e99077c522de47cf9e589d9109a19ce8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1b8b6260af84d639e003abc3bdab0f1"
I20260810 14:53:13.165047 32672 raft_consensus.cc:3037] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:13.165050 32522 raft_consensus.cc:3037] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:13.165663 32522 raft_consensus.cc:2454] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e99077c522de47cf9e589d9109a19ce8 in term 1.
I20260810 14:53:13.165668 32672 raft_consensus.cc:2454] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e99077c522de47cf9e589d9109a19ce8 in term 1.
I20260810 14:53:13.165864 32305 leader_election.cc:302] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e176dd8920184f0584480f93ed9968b3, e99077c522de47cf9e589d9109a19ce8; no voters:
I20260810 14:53:13.166002 32756 raft_consensus.cc:2781] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:13.166082 32756 raft_consensus.cc:686] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [term 1 LEADER]: Becoming Leader. State: Replica: e99077c522de47cf9e589d9109a19ce8, State: Running, Role: LEADER
I20260810 14:53:13.166205 32756 consensus_queue.cc:237] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } }
I20260810 14:53:13.166554 310 tablet_replica.cc:439] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8: TxnStatusTablet state changed. Reason: New leader e99077c522de47cf9e589d9109a19ce8. Latest consensus state: current_term: 1 leader_uuid: "e99077c522de47cf9e589d9109a19ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } }
I20260810 14:53:13.166657 310 tablet_replica.cc:442] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:53:13.166915 323 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 14:53:13.166929 32211 catalog_manager.cc:5495] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e99077c522de47cf9e589d9109a19ce8 (127.30.74.65). New cstate: current_term: 1 leader_uuid: "e99077c522de47cf9e589d9109a19ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } health_report { overall_health: UNKNOWN } } }
I20260810 14:53:13.167055 323 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 14:53:13.176470 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:13.173276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53986 (local address 127.30.74.65:38183)
0810 14:53:13.173328 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:13.173330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:13.173341 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:53:13.173403 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:13.173404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:13.173426 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:13.173504 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:13.173589 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.174412 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.175510 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.175813 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.175889 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:13.176097 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:13.176214 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:13.176324 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:13.176373 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 14:53:13.176397 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 14:53:13.176505 317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:13.173182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:38183 (local address 127.0.0.1:53986)
0810 14:53:13.173266 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:13.173280 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:13.173342 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:13.173503 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:13.173504 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:13.173505 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:53:13.173551 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:13.173552 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.174501 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.174503 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:13.175406 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:13.175410 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.175904 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.175906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:13.175972 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:13.175991 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:53:13.176309 (+ 318us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:53:13.176319 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:53:13.176370 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 14:53:13.176428 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 14:53:13.177826 32522 raft_consensus.cc:1261] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3 [term 1 FOLLOWER]: Refusing update from remote peer e99077c522de47cf9e589d9109a19ce8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:13.177837 32672 raft_consensus.cc:1261] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1 [term 1 FOLLOWER]: Refusing update from remote peer e99077c522de47cf9e589d9109a19ce8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:13.178103 32756 consensus_queue.cc:1035] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:13.178257 32756 consensus_queue.cc:1035] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:13.178874 32755 tablet_replica.cc:439] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3: TxnStatusTablet state changed. Reason: New leader e99077c522de47cf9e589d9109a19ce8. Latest consensus state: current_term: 1 leader_uuid: "e99077c522de47cf9e589d9109a19ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } }
I20260810 14:53:13.178967 32755 tablet_replica.cc:442] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:13.178915 32754 tablet_replica.cc:439] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1: TxnStatusTablet state changed. Reason: New leader e99077c522de47cf9e589d9109a19ce8. Latest consensus state: current_term: 1 leader_uuid: "e99077c522de47cf9e589d9109a19ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } }
I20260810 14:53:13.179010 32754 tablet_replica.cc:442] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:13.179561 32756 tablet_replica.cc:439] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e99077c522de47cf9e589d9109a19ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } }
I20260810 14:53:13.179641 32756 tablet_replica.cc:442] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:53:13.179718 32762 mvcc.cc:205] Tried to move back new op lower bound from 7316986237654372352 to 7316986237609226240. Current Snapshot: MvccSnapshot[applied={T|T < 7316986237654372352 or (T in {7316986237654372352})}]
I20260810 14:53:13.180001 310 tablet_replica.cc:439] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e99077c522de47cf9e589d9109a19ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } }
I20260810 14:53:13.180110 310 tablet_replica.cc:442] T 228541ff58074c978698183453e16041 P e99077c522de47cf9e589d9109a19ce8: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:53:13.180292 32755 tablet_replica.cc:439] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e99077c522de47cf9e589d9109a19ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } }
I20260810 14:53:13.180377 32755 tablet_replica.cc:442] T 228541ff58074c978698183453e16041 P e176dd8920184f0584480f93ed9968b3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:13.180382 32754 tablet_replica.cc:439] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e99077c522de47cf9e589d9109a19ce8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e99077c522de47cf9e589d9109a19ce8" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38183 } } peers { permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 } } peers { permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 } } }
I20260810 14:53:13.180467 32754 tablet_replica.cc:442] T 228541ff58074c978698183453e16041 P e1b8b6260af84d639e003abc3bdab0f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:13.180476 32761 mvcc.cc:205] Tried to move back new op lower bound from 7316986237654372352 to 7316986237609226240. Current Snapshot: MvccSnapshot[applied={T|T < 7316986237654372352 or (T in {7316986237654372352})}]
I20260810 14:53:13.184656 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:13.181777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53996 (local address 127.30.74.65:38183)
0810 14:53:13.181816 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:13.181818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:13.181827 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:13.181892 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:13.181893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:13.181904 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:13.181978 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:13.182132 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.182862 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.183925 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.184107 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.184164 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:13.184411 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:13.184463 (+ 52us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:13.184508 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:13.184592 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 14:53:13.184608 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 14:53:13.189785 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:13.187199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:41047 (local address 127.30.74.65:38183)
0810 14:53:13.187254 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:13.187256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:13.187273 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:53:13.187334 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:13.187335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:13.187344 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:13.187402 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:13.187531 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.187958 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.189115 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.189290 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.189348 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:13.189534 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:13.189557 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:13.189592 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:13.189669 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 14:53:13.189712 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":33}
I20260810 14:53:13.189956 331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:13.187040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:38183 (local address 127.30.74.65:41047)
0810 14:53:13.187204 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:13.187216 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:13.187274 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:13.187406 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:13.187407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:13.187408 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:53:13.187473 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:13.187474 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.188030 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.188031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:13.189032 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:13.189035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.189366 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.189368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:13.189435 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:13.189450 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:53:13.189613 (+ 163us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:53:13.189615 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 14:53:13.189668 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 14:53:13.189726 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":4,"thread_start_us":78,"threads_started":1}
I20260810 14:53:13.192044 32672 raft_consensus.cc:1261] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1 [term 1 FOLLOWER]: Refusing update from remote peer e99077c522de47cf9e589d9109a19ce8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:13.192142 32522 raft_consensus.cc:1261] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3 [term 1 FOLLOWER]: Refusing update from remote peer e99077c522de47cf9e589d9109a19ce8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:13.192373 32756 consensus_queue.cc:1035] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1b8b6260af84d639e003abc3bdab0f1" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:13.192478 310 consensus_queue.cc:1035] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e176dd8920184f0584480f93ed9968b3" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 37583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:13.205168 32212 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:13.205307 32212 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:13.205988 32212 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b46c1f63b36448d988de3d3e410dd9a9: Sending DeleteTablet for 3 replicas of tablet 1866930e9fa647318c211b0006d50e04
I20260810 14:53:13.206354 32650 tablet_service.cc:1514] Processing DeleteTablet for tablet 1866930e9fa647318c211b0006d50e04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:13 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:54344
I20260810 14:53:13.206423 32501 tablet_service.cc:1514] Processing DeleteTablet for tablet 1866930e9fa647318c211b0006d50e04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:13 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:39296
I20260810 14:53:13.206514 302 tablet_replica.cc:330] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3: stopping tablet replica
I20260810 14:53:13.206498 303 tablet_replica.cc:330] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1: stopping tablet replica
I20260810 14:53:13.206571 302 raft_consensus.cc:2229] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:13.206571 303 raft_consensus.cc:2229] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:13.206609 302 raft_consensus.cc:2258] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:13.206615 303 raft_consensus.cc:2258] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:13.206768 32354 tablet_service.cc:1514] Processing DeleteTablet for tablet 1866930e9fa647318c211b0006d50e04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:13 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:53978
I20260810 14:53:13.206868 304 tablet_replica.cc:330] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8: stopping tablet replica
I20260810 14:53:13.206926 304 raft_consensus.cc:2229] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:53:13.206980 303 ts_tablet_manager.cc:1905] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:13.207020 304 raft_consensus.cc:2258] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:13.207345 304 ts_tablet_manager.cc:1905] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:13.207644 302 ts_tablet_manager.cc:1905] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:13.208757 303 ts_tablet_manager.cc:1918] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:53:13.208833 303 log.cc:1196] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/1866930e9fa647318c211b0006d50e04
I20260810 14:53:13.209070 303 ts_tablet_manager.cc:1939] T 1866930e9fa647318c211b0006d50e04 P e1b8b6260af84d639e003abc3bdab0f1: Deleting consensus metadata
I20260810 14:53:13.209085 304 ts_tablet_manager.cc:1918] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:53:13.209153 304 log.cc:1196] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-0/wal/wals/1866930e9fa647318c211b0006d50e04
I20260810 14:53:13.209311 302 ts_tablet_manager.cc:1918] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:53:13.209380 302 log.cc:1196] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/1866930e9fa647318c211b0006d50e04
I20260810 14:53:13.209393 304 ts_tablet_manager.cc:1939] T 1866930e9fa647318c211b0006d50e04 P e99077c522de47cf9e589d9109a19ce8: Deleting consensus metadata
I20260810 14:53:13.209457 32198 catalog_manager.cc:4841] TS e1b8b6260af84d639e003abc3bdab0f1 (127.30.74.67:42785): tablet 1866930e9fa647318c211b0006d50e04 (table test-table [id=819152da626c42569a7c4fbdd7af2109]) successfully deleted
I20260810 14:53:13.209586 302 ts_tablet_manager.cc:1939] T 1866930e9fa647318c211b0006d50e04 P e176dd8920184f0584480f93ed9968b3: Deleting consensus metadata
I20260810 14:53:13.209741 32199 catalog_manager.cc:4841] TS e99077c522de47cf9e589d9109a19ce8 (127.30.74.65:38183): tablet 1866930e9fa647318c211b0006d50e04 (table test-table [id=819152da626c42569a7c4fbdd7af2109]) successfully deleted
I20260810 14:53:13.209875 32197 catalog_manager.cc:4841] TS e176dd8920184f0584480f93ed9968b3 (127.30.74.66:37583): tablet 1866930e9fa647318c211b0006d50e04 (table test-table [id=819152da626c42569a7c4fbdd7af2109]) successfully deleted
Aug 10 14:53:13 dist-test-slave-0b58 krb5kdc[32164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373592, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.74.65@KRBTEST.COM
I20260810 14:53:13.222162 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:13.212586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54010 (local address 127.30.74.65:38183)
0810 14:53:13.212638 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:13.212640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:13.212808 (+ 168us) server_negotiation.cc:407] Connection header received
0810 14:53:13.212866 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:13.212867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:13.212877 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:13.212925 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:13.213655 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.214164 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.215274 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.215460 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.215524 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:13.218893 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:13.218905 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:13.218905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:13.218914 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:13.220903 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:13.221281 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:13.221285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:13.221285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:13.221365 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:13.221573 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:13.221574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:13.221574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:13.221733 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:13.221918 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:13.221956 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 14:53:13.222038 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260810 14:53:13.222543 32413 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54010
I20260810 14:53:13.230506 318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:13.223555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58512 (local address 127.30.74.126:39763)
0810 14:53:13.223602 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:13.223604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:13.223678 (+ 74us) server_negotiation.cc:407] Connection header received
0810 14:53:13.223750 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:13.223751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:13.223761 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:13.223815 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:13.224482 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.225010 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.226087 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:13.226312 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:13.226377 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:13.227796 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:13.227798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:13.227798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:13.227804 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:13.229321 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:13.229727 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:13.229728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:13.229729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:13.229793 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:13.230013 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:13.230013 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:13.230014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:13.230135 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:13.230217 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:13.230354 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 14:53:13.230438 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 14:53:13.230931 32212 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58512
I20260810 14:53:13.231643 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 32287
I20260810 14:53:13.241364 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 32432
I20260810 14:53:13.249890 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 32580
I20260810 14:53:13.257344 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 32180
2026-08-10T14:53:13Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2635 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:53:13 dist-test-slave-0b58 krb5kdc[345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:53:13 dist-test-slave-0b58 krb5kdc[345](info): set up 2 sockets
Aug 10 14:53:13 dist-test-slave-0b58 krb5kdc[345](info): commencing operation
krb5kdc: starting...
W20260810 14:53:14.346719 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:53:14 dist-test-slave-0b58 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:53:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:53:14.482358 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:41085
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:41085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:14.565179 359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:14.565447 359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:14.565507 359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:14.566792 359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:14.566866 359 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 14:53:14.566887 359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:14.566910 359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:14.566942 359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:14.566967 359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:14.568614 359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:41085
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:41085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:14.568917 359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:14.569160 359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:14.574221 367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:14.574216 365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:14.574396 359 server_base.cc:1034] running on GCE node
W20260810 14:53:14.574240 364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:14.574860 359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:14.575178 359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:14.576340 359 hybrid_clock.cc:648] HybridClock initialized: now 1786373594576309 us; error 36 us; skew 500 ppm
Aug 10 14:53:14 dist-test-slave-0b58 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373594, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:14.580355 359 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:53:14.580832 359 webserver.cc:460] Webserver started at http://127.30.74.126:42513/ using document root <none> and password file <none>
I20260810 14:53:14.581040 359 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:14.581084 359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:14.581172 359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:14.582221 359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "9085c84712ab4adba9211ee9b47216f8"
format_stamp: "Formatted at 2026-08-10 14:53:14 on dist-test-slave-0b58"
server_key: "4cd18bcb3afb0e9a045c860482c07ddc"
server_key_iv: "cf7372f901ce5254a7a3b27de5f256e9"
server_key_version: "encryptionkey@0"
I20260810 14:53:14.582536 359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "9085c84712ab4adba9211ee9b47216f8"
format_stamp: "Formatted at 2026-08-10 14:53:14 on dist-test-slave-0b58"
server_key: "4cd18bcb3afb0e9a045c860482c07ddc"
server_key_iv: "cf7372f901ce5254a7a3b27de5f256e9"
server_key_version: "encryptionkey@0"
I20260810 14:53:14.583825 359 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:53:14.584578 375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:14.584751 359 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:14.584842 359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "9085c84712ab4adba9211ee9b47216f8"
format_stamp: "Formatted at 2026-08-10 14:53:14 on dist-test-slave-0b58"
server_key: "4cd18bcb3afb0e9a045c860482c07ddc"
server_key_iv: "cf7372f901ce5254a7a3b27de5f256e9"
server_key_version: "encryptionkey@0"
I20260810 14:53:14.584901 359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:14.596088 359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:14.598276 359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:14.598397 359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:14.603005 359 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:41085
I20260810 14:53:14.603183 441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:41085 every 8 connection(s)
I20260810 14:53:14.603518 359 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:14.604308 442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:14.606624 442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8: Bootstrap starting.
I20260810 14:53:14.607192 442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:14.607434 442 log.cc:824] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:14.607699 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 359
I20260810 14:53:14.607789 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:14.607981 31017 external_mini_cluster.cc:1428] Setting key 66fba1e110d124b02e76ac2ea8ea57f6
I20260810 14:53:14.608346 442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8: No bootstrap required, opened a new log
I20260810 14:53:14.610615 442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9085c84712ab4adba9211ee9b47216f8" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41085 } }
I20260810 14:53:14.610793 442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:14.610842 442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9085c84712ab4adba9211ee9b47216f8, State: Initialized, Role: FOLLOWER
I20260810 14:53:14.610977 442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9085c84712ab4adba9211ee9b47216f8" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41085 } }
I20260810 14:53:14.611068 442 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:14.611115 442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:14.611168 442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:14.611718 442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9085c84712ab4adba9211ee9b47216f8" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41085 } }
I20260810 14:53:14.611846 442 leader_election.cc:302] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9085c84712ab4adba9211ee9b47216f8; no voters:
I20260810 14:53:14.612066 442 leader_election.cc:288] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:14.612098 448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:14.612226 448 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [term 1 LEADER]: Becoming Leader. State: Replica: 9085c84712ab4adba9211ee9b47216f8, State: Running, Role: LEADER
I20260810 14:53:14.612340 448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9085c84712ab4adba9211ee9b47216f8" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41085 } }
I20260810 14:53:14.612383 442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:14.612704 448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9085c84712ab4adba9211ee9b47216f8. Latest consensus state: current_term: 1 leader_uuid: "9085c84712ab4adba9211ee9b47216f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9085c84712ab4adba9211ee9b47216f8" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41085 } } }
I20260810 14:53:14.612778 448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:14.613122 450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9085c84712ab4adba9211ee9b47216f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9085c84712ab4adba9211ee9b47216f8" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41085 } } }
I20260810 14:53:14.613216 450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:14.613336 455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:14.613888 455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:14.615967 455 catalog_manager.cc:1347] Generated new cluster ID: b8f4367ed26f43f798d530b612dc5322
I20260810 14:53:14.616030 455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 14:53:14 dist-test-slave-0b58 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:14.622471 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:14.608551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41648 (local address 127.30.74.126:41085)
0810 14:53:14.608757 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:14.608782 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:14.608819 (+ 37us) server_negotiation.cc:407] Connection header received
0810 14:53:14.610596 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:14.610604 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:14.610656 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:14.610817 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:14.611566 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.613515 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.614656 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.615097 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.615176 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:14.618218 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:14.618222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:14.618222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:14.618235 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:14.620773 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:14.621193 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:14.621196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:14.621196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:14.621309 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:14.621528 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:14.621529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:14.621529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:14.621842 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:14.621989 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:14.622123 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 14:53:14.622263 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":92,"threads_started":1}
I20260810 14:53:14.623472 455 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:14.623998 455 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:14.628810 455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8: Generated new TSK 0
I20260810 14:53:14.628999 455 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:53:14.656914 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:41085
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:14.740415 470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:14.740645 470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:14.740689 470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:14.741977 470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:14.742048 470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:14.742081 470 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 14:53:14.742108 470 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:53:14.742146 470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:53:14.743682 470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.74.126:41085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:14.743966 470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:14.744208 470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:14.749125 476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:14.749145 477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:14.749205 479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:14.749457 470 server_base.cc:1034] running on GCE node
I20260810 14:53:14.749632 470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:14.749850 470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:14.751020 470 hybrid_clock.cc:648] HybridClock initialized: now 1786373594751011 us; error 27 us; skew 500 ppm
Aug 10 14:53:14 dist-test-slave-0b58 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373594, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:14.753628 470 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:53:14.754089 470 webserver.cc:460] Webserver started at http://127.30.74.65:35949/ using document root <none> and password file <none>
I20260810 14:53:14.754338 470 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:14.754410 470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:14.754575 470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:14.755631 470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "8cf6044aea834d71be137a3e7936b43d"
format_stamp: "Formatted at 2026-08-10 14:53:14 on dist-test-slave-0b58"
server_key: "7c9ff2595608173a4164cfd83ad5440d"
server_key_iv: "27ff98144de95508aeb0d8588e3e6d71"
server_key_version: "encryptionkey@0"
I20260810 14:53:14.756011 470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "8cf6044aea834d71be137a3e7936b43d"
format_stamp: "Formatted at 2026-08-10 14:53:14 on dist-test-slave-0b58"
server_key: "7c9ff2595608173a4164cfd83ad5440d"
server_key_iv: "27ff98144de95508aeb0d8588e3e6d71"
server_key_version: "encryptionkey@0"
I20260810 14:53:14.757428 470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:53:14.758212 487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:14.758384 470 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:14.758477 470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "8cf6044aea834d71be137a3e7936b43d"
format_stamp: "Formatted at 2026-08-10 14:53:14 on dist-test-slave-0b58"
server_key: "7c9ff2595608173a4164cfd83ad5440d"
server_key_iv: "27ff98144de95508aeb0d8588e3e6d71"
server_key_version: "encryptionkey@0"
I20260810 14:53:14.758560 470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:14.769594 470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:14.771757 470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:14.771931 470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:14.772600 470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:14.772665 470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:14.772696 470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:14.772722 470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:53:14 dist-test-slave-0b58 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373594, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:14.779928 470 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:38517
I20260810 14:53:14.780018 607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:38517 every 8 connection(s)
I20260810 14:53:14.780308 470 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:53:14.782581 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 470
I20260810 14:53:14.782685 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:53:14.782729 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:14.773002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:59285 (local address 127.30.74.126:41085)
0810 14:53:14.773072 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:14.773075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:14.773318 (+ 243us) server_negotiation.cc:407] Connection header received
0810 14:53:14.773850 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:14.773851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:14.773868 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:14.773931 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:14.774801 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.775434 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.776806 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.777017 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.777105 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:14.779717 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:14.779718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:14.779719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:14.779727 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:14.781450 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:14.781816 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:14.781816 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:14.781817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:14.781867 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:14.782074 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:14.782074 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:14.782074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:14.782177 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:14.782256 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:14.782544 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 14:53:14.782637 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 14:53:14.782899 31017 external_mini_cluster.cc:1428] Setting key 56b5d8737c223d106b4ee5f210ff6e27
I20260810 14:53:14.782896 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:14.773003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:41085 (local address 127.30.74.65:59285)
0810 14:53:14.773181 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:14.773222 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:14.773742 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:14.774009 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:14.774013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:14.774381 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:14.774704 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:14.774711 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.775528 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.775529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:14.776712 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:14.776715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.777132 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.777133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:14.777260 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:14.777826 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:14.777834 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:14.779587 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:14.781526 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:14.781530 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:14.781532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:14.781743 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:14.781926 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:14.781926 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:14.781926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:14.782021 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:14.782280 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:14.782284 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:14.782396 (+ 112us) client_negotiation.cc:769] Sending connection context
0810 14:53:14.782514 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 14:53:14.782671 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
I20260810 14:53:14.783557 608 heartbeater.cc:347] Connected to a master server at 127.30.74.126:41085
I20260810 14:53:14.783653 608 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:14.783861 608 heartbeater.cc:511] Master 127.30.74.126:41085 requested a full tablet report, sending...
I20260810 14:53:14.785030 392 ts_manager.cc:194] Registered new tserver with Master: 8cf6044aea834d71be137a3e7936b43d (127.30.74.65:38517)
I20260810 14:53:14.785570 392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:59285
I20260810 14:53:14.792052 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:14.784756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:49063 (local address 127.30.74.126:41085)
0810 14:53:14.784799 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:14.784801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:14.785895 (+ 1094us) server_negotiation.cc:407] Connection header received
0810 14:53:14.786000 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:14.786001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:14.786013 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:14.786044 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:14.786519 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.787053 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.788129 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.788476 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.788546 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:14.789576 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:14.789578 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:14.789578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:14.789584 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:14.790964 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:14.791372 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:14.791373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:14.791373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:14.791421 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:14.791624 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:14.791625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:14.791625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:14.791717 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:14.791766 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:14.791935 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 14:53:14.791993 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 14:53:14.792455 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:14.784735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:41085 (local address 127.30.74.65:49063)
0810 14:53:14.785820 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:14.785831 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:14.785923 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:14.786127 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:14.786127 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:14.786358 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:14.786436 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:14.786438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.787153 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.787154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:14.788037 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:14.788040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.788557 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.788559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:14.788635 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:14.789066 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:14.789071 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:14.789485 (+ 414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:14.791053 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:14.791055 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:14.791055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:14.791289 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:14.791488 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:14.791489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:14.791489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:14.791558 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:14.791782 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:14.791783 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:14.791833 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 14:53:14.791921 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 14:53:14.792023 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1058,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:53:14.811085 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:41085
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:14.903198 619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:14.903573 619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:14.903625 619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:14.905450 619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:14.905507 619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:14.905546 619 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 14:53:14.905571 619 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:53:14.905606 619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:53:14.908249 619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.74.126:41085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:14.908565 619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:14.908895 619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:14.915484 625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:14.915546 626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:14.915469 628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:14.915932 619 server_base.cc:1034] running on GCE node
I20260810 14:53:14.916179 619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:14.916482 619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:14.917693 619 hybrid_clock.cc:648] HybridClock initialized: now 1786373594917672 us; error 35 us; skew 500 ppm
Aug 10 14:53:14 dist-test-slave-0b58 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373594, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:14.921159 619 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:53:14.921617 619 webserver.cc:460] Webserver started at http://127.30.74.66:37245/ using document root <none> and password file <none>
I20260810 14:53:14.921900 619 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:14.921977 619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:14.922119 619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:14.923220 619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "9b6fe2f6be6a45f699c3534e102d4dcf"
format_stamp: "Formatted at 2026-08-10 14:53:14 on dist-test-slave-0b58"
server_key: "98b5dc6ddaf15284c0527893def0c903"
server_key_iv: "f5c890b6f7f7f032d0f85ba62565402d"
server_key_version: "encryptionkey@0"
I20260810 14:53:14.923597 619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "9b6fe2f6be6a45f699c3534e102d4dcf"
format_stamp: "Formatted at 2026-08-10 14:53:14 on dist-test-slave-0b58"
server_key: "98b5dc6ddaf15284c0527893def0c903"
server_key_iv: "f5c890b6f7f7f032d0f85ba62565402d"
server_key_version: "encryptionkey@0"
I20260810 14:53:14.925117 619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260810 14:53:14.925956 636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:14.926167 619 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:14.926277 619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "9b6fe2f6be6a45f699c3534e102d4dcf"
format_stamp: "Formatted at 2026-08-10 14:53:14 on dist-test-slave-0b58"
server_key: "98b5dc6ddaf15284c0527893def0c903"
server_key_iv: "f5c890b6f7f7f032d0f85ba62565402d"
server_key_version: "encryptionkey@0"
I20260810 14:53:14.926354 619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:14.940615 619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:14.942994 619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:14.943161 619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:14.943691 619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:14.943756 619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:14.943831 619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:14.943862 619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:53:14 dist-test-slave-0b58 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373594, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:14.951290 619 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:34207
I20260810 14:53:14.951397 758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:34207 every 8 connection(s)
I20260810 14:53:14.951718 619 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:53:14.954165 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:14.944243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:33563 (local address 127.30.74.126:41085)
0810 14:53:14.944376 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:14.944379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:14.944587 (+ 208us) server_negotiation.cc:407] Connection header received
0810 14:53:14.945098 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:14.945099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:14.945115 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:14.945199 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:14.946035 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.946881 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.948576 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.948823 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.948911 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:14.951275 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:14.951277 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:14.951277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:14.951285 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:14.952981 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:14.953357 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:14.953358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:14.953358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:14.953413 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:14.953593 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:14.953593 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:14.953594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:14.953702 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:14.953754 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:14.954000 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 14:53:14.954065 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260810 14:53:14.954241 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:14.944242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:41085 (local address 127.30.74.66:33563)
0810 14:53:14.944453 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:14.944495 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:14.944999 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:14.945307 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:14.945311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:14.945611 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:14.945912 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:14.945920 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.947003 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.947004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:14.948459 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:14.948463 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.948934 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.948936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:14.949104 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:14.949636 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:14.949650 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:14.951168 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:14.953060 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:14.953063 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:14.953065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:14.953281 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:14.953460 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:14.953460 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:14.953460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:14.953542 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:14.953771 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:14.953776 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:14.953889 (+ 113us) client_negotiation.cc:769] Sending connection context
0810 14:53:14.953971 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 14:53:14.954091 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":139,"threads_started":1}
I20260810 14:53:14.954804 759 heartbeater.cc:347] Connected to a master server at 127.30.74.126:41085
I20260810 14:53:14.954890 759 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:14.955072 759 heartbeater.cc:511] Master 127.30.74.126:41085 requested a full tablet report, sending...
I20260810 14:53:14.955508 390 ts_manager.cc:194] Registered new tserver with Master: 9b6fe2f6be6a45f699c3534e102d4dcf (127.30.74.66:34207)
I20260810 14:53:14.955992 390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:33563
I20260810 14:53:14.957278 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 619
I20260810 14:53:14.957382 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:53:14.957572 31017 external_mini_cluster.cc:1428] Setting key b29ff647f0db78aeea7852b9f4dae329
I20260810 14:53:14.962215 765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:14.955861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:41085 (local address 127.30.74.66:54383)
0810 14:53:14.956032 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:14.956041 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:14.956195 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:14.956377 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:14.956378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:14.956541 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:14.956596 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:14.956597 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.957282 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.957283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:14.958071 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:14.958073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.958510 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.958511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:14.958601 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:14.958984 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:14.958990 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:14.959401 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:14.960982 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:14.960983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:14.960984 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:14.961255 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:14.961518 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:14.961519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:14.961519 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:14.961595 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:14.961967 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:14.961968 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:14.962007 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 14:53:14.962060 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 14:53:14.962146 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260810 14:53:14.962364 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:14.955924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:54383 (local address 127.30.74.126:41085)
0810 14:53:14.955968 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:14.955970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:14.956164 (+ 194us) server_negotiation.cc:407] Connection header received
0810 14:53:14.956251 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:14.956251 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:14.956304 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:14.956358 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:14.956674 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.957177 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.958154 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:14.958424 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:14.958485 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:14.959488 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:14.959489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:14.959490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:14.959495 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:14.960894 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:14.961353 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:14.961355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:14.961355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:14.961417 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:14.961671 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:14.961672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:14.961672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:14.961897 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:14.961970 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:14.962101 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 14:53:14.962185 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:53:14.986681 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:41085
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:15.072851 769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:15.073102 769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:15.073129 769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:15.074271 769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:15.074308 769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:15.074328 769 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 14:53:15.074340 769 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:53:15.074357 769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:53:15.075783 769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.74.126:41085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:15.075979 769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:15.076152 769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:15.081184 774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:15.081166 775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:15.081293 777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:15.081552 769 server_base.cc:1034] running on GCE node
I20260810 14:53:15.081802 769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:15.082055 769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:15.083250 769 hybrid_clock.cc:648] HybridClock initialized: now 1786373595083242 us; error 33 us; skew 500 ppm
Aug 10 14:53:15 dist-test-slave-0b58 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373595, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:15.086099 769 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:53:15.086544 769 webserver.cc:460] Webserver started at http://127.30.74.67:34005/ using document root <none> and password file <none>
I20260810 14:53:15.086810 769 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:15.086881 769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:15.087033 769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:15.088186 769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "493b4c03e00a4783aef988e4b268237f"
format_stamp: "Formatted at 2026-08-10 14:53:15 on dist-test-slave-0b58"
server_key: "493e291cf1247099960fdddfa13800e0"
server_key_iv: "77555aac1b2b8846a61a384534dcba99"
server_key_version: "encryptionkey@0"
I20260810 14:53:15.088649 769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "493b4c03e00a4783aef988e4b268237f"
format_stamp: "Formatted at 2026-08-10 14:53:15 on dist-test-slave-0b58"
server_key: "493e291cf1247099960fdddfa13800e0"
server_key_iv: "77555aac1b2b8846a61a384534dcba99"
server_key_version: "encryptionkey@0"
I20260810 14:53:15.090170 769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:15.090978 784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:15.091147 769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:15.091257 769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "493b4c03e00a4783aef988e4b268237f"
format_stamp: "Formatted at 2026-08-10 14:53:15 on dist-test-slave-0b58"
server_key: "493e291cf1247099960fdddfa13800e0"
server_key_iv: "77555aac1b2b8846a61a384534dcba99"
server_key_version: "encryptionkey@0"
I20260810 14:53:15.091336 769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:15.101975 769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:15.104409 769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:15.104583 769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:15.105129 769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:15.105195 769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:15.105242 769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:15.105269 769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:53:15 dist-test-slave-0b58 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373595, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:15.113226 769 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:38345
I20260810 14:53:15.113379 907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:38345 every 8 connection(s)
I20260810 14:53:15.113783 769 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:53:15.115813 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.105683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:39777 (local address 127.30.74.126:41085)
0810 14:53:15.105758 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:15.105761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:15.106006 (+ 245us) server_negotiation.cc:407] Connection header received
0810 14:53:15.106805 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:15.106808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:15.106824 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:15.106863 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:15.107949 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.108616 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.109669 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.109869 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.109950 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.112795 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:15.112796 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:15.112797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:15.112804 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:15.114597 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:15.114964 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:15.114964 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:15.114965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:15.115014 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:15.115227 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:15.115228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:15.115228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:15.115368 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:15.115436 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:15.115676 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 14:53:15.115748 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 14:53:15.115952 795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.105645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:41085 (local address 127.30.74.67:39777)
0810 14:53:15.105854 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:15.105905 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:15.106654 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:15.107074 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:15.107079 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:15.107509 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:15.107835 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.107843 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.108729 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.108731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.109558 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.109560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.109981 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.109983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.110103 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.110651 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:15.110667 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:15.112702 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:15.114672 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:15.114676 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:15.114678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:15.114897 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:15.115054 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:15.115055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:15.115055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:15.115142 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:15.115440 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:15.115445 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:15.115560 (+ 115us) client_negotiation.cc:769] Sending connection context
0810 14:53:15.115642 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 14:53:15.115764 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260810 14:53:15.116519 908 heartbeater.cc:347] Connected to a master server at 127.30.74.126:41085
I20260810 14:53:15.116626 908 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:15.116806 908 heartbeater.cc:511] Master 127.30.74.126:41085 requested a full tablet report, sending...
I20260810 14:53:15.117233 390 ts_manager.cc:194] Registered new tserver with Master: 493b4c03e00a4783aef988e4b268237f (127.30.74.67:38345)
I20260810 14:53:15.117693 390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:39777
I20260810 14:53:15.122611 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 769
I20260810 14:53:15.122714 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:53:15.122927 31017 external_mini_cluster.cc:1428] Setting key 63140336db0e5ab3bc25f7f58b122aca
I20260810 14:53:15.123521 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.117743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:55667 (local address 127.30.74.126:41085)
0810 14:53:15.117789 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:15.117791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:15.117875 (+ 84us) server_negotiation.cc:407] Connection header received
0810 14:53:15.117932 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:15.117932 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:15.117943 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:15.117999 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:15.118350 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.119106 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.119822 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.120033 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.120123 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.120995 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:15.120997 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:15.120997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:15.121004 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:15.122176 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:15.122667 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:15.122668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:15.122668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:15.122756 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:15.122997 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:15.122998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:15.122998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:15.123111 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:15.123182 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:15.123350 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 14:53:15.123430 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 14:53:15.123528 914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.117672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:41085 (local address 127.30.74.67:55667)
0810 14:53:15.117813 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:15.117825 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:15.117882 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:15.118018 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:15.118019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:15.118177 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:15.118255 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.118257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.119214 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.119215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.119736 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.119738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.120127 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.120128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.120214 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.120636 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:15.120640 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:15.120904 (+ 264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:15.122299 (+ 1395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:15.122300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:15.122301 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:15.122544 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:15.122846 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:15.122847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:15.122847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:15.122919 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:15.123187 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:15.123188 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:15.123233 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 14:53:15.123324 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 14:53:15.123432 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":121,"threads_started":1}
I20260810 14:53:15.124282 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:53:15 dist-test-slave-0b58 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 14:53:15 dist-test-slave-0b58 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:15.148599 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.141388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41650 (local address 127.30.74.126:41085)
0810 14:53:15.141552 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:15.141554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:15.141565 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:53:15.141602 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:15.141603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:15.141612 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:15.141664 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:15.142244 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.142748 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.143489 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.143663 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.143711 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.146020 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:15.146022 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:15.146022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:15.146038 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:15.147500 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:15.147891 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:15.147892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:15.147892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:15.147946 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:15.148119 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:15.148120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:15.148120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:15.148225 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:15.148354 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:15.148467 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 14:53:15.148529 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260810 14:53:15.150583 390 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:53:15.152179 390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:15.158937 931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.154893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42336 (local address 127.30.74.65:38517)
0810 14:53:15.155073 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:15.155078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:15.155086 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:53:15.155122 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:15.155123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:15.155135 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:15.155223 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:15.155446 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.156612 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.157866 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.158463 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.158557 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.158620 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:15.158790 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 14:53:15.158831 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":98,"threads_started":1}
I20260810 14:53:15.158928 927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.154607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:38517 (local address 127.0.0.1:42336)
0810 14:53:15.154959 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:15.154972 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:15.155028 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:15.155263 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:15.155267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:15.155269 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:15.155345 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.155347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.156721 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.156722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.157779 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.157781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.158556 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.158557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.158646 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.158662 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 14:53:15.158771 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 14:53:15.158832 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":35,"thread_start_us":111,"threads_started":1}
I20260810 14:53:15.159515 929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.154597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:34207 (local address 127.0.0.1:34872)
0810 14:53:15.154906 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:15.154915 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:15.154955 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:15.155470 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:15.155471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:15.155473 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:15.155538 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.155540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.156655 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.156656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.157974 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.157978 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.158958 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.158958 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.159359 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.159365 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:53:15.159405 (+ 40us) client_negotiation.cc:240] Negotiation successful
0810 14:53:15.159445 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":1,"thread_start_us":131,"threads_started":1}
I20260810 14:53:15.159849 928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.154591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:38345 (local address 127.0.0.1:54136)
0810 14:53:15.154773 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:15.154817 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:15.154894 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:15.155541 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:15.155541 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:15.155543 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:15.155581 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.155582 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.156926 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.156927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.157981 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.157982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.158956 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.158957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.159685 (+ 728us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.159688 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:53:15.159746 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 14:53:15.159787 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":662,"thread_start_us":61,"threads_started":1}
I20260810 14:53:15.159734 539 tablet_service.cc:1467] Processing CreateTablet for tablet 87d4122ed7e94f879003d2f97164399f (DEFAULT_TABLE table=test-table [id=dde1538ebfbf4c259840b54fb1dae8ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:15.160174 539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87d4122ed7e94f879003d2f97164399f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:15.160315 932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.155102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34872 (local address 127.30.74.66:34207)
0810 14:53:15.155298 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:15.155302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:15.155312 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:15.155351 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:15.155353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:15.155369 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:15.155619 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:15.155684 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.156565 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.158087 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.158842 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.160035 (+ 1193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.160108 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:15.160136 (+ 28us) server_negotiation.cc:299] Negotiation successful
0810 14:53:15.160183 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":110,"threads_started":1}
I20260810 14:53:15.161011 678 tablet_service.cc:1467] Processing CreateTablet for tablet 87d4122ed7e94f879003d2f97164399f (DEFAULT_TABLE table=test-table [id=dde1538ebfbf4c259840b54fb1dae8ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:15.161373 678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87d4122ed7e94f879003d2f97164399f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:15.164414 933 tablet_bootstrap.cc:492] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d: Bootstrap starting.
I20260810 14:53:15.164628 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.154780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54136 (local address 127.30.74.67:38345)
0810 14:53:15.155354 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:15.155359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:15.155368 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:15.155409 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:15.155411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:15.155428 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:15.155652 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:15.155729 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.156844 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.158087 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.158842 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.158980 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.159052 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:15.164392 (+ 5340us) server_negotiation.cc:299] Negotiation successful
0810 14:53:15.164460 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":87,"threads_started":1}
I20260810 14:53:15.165124 933 tablet_bootstrap.cc:654] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:15.165305 934 tablet_bootstrap.cc:492] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf: Bootstrap starting.
I20260810 14:53:15.165313 838 tablet_service.cc:1467] Processing CreateTablet for tablet 87d4122ed7e94f879003d2f97164399f (DEFAULT_TABLE table=test-table [id=dde1538ebfbf4c259840b54fb1dae8ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:15.165493 933 log.cc:824] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:15.165632 838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87d4122ed7e94f879003d2f97164399f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:15.165946 934 tablet_bootstrap.cc:654] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:15.166447 933 tablet_bootstrap.cc:492] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d: No bootstrap required, opened a new log
I20260810 14:53:15.166512 933 ts_tablet_manager.cc:1397] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 14:53:15.166497 934 log.cc:824] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:15.167320 934 tablet_bootstrap.cc:492] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf: No bootstrap required, opened a new log
I20260810 14:53:15.167377 934 ts_tablet_manager.cc:1397] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:53:15.168488 937 tablet_bootstrap.cc:492] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f: Bootstrap starting.
I20260810 14:53:15.168617 933 raft_consensus.cc:348] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } }
I20260810 14:53:15.168803 933 raft_consensus.cc:374] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:15.168840 933 raft_consensus.cc:729] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cf6044aea834d71be137a3e7936b43d, State: Initialized, Role: FOLLOWER
I20260810 14:53:15.168991 933 consensus_queue.cc:260] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } }
I20260810 14:53:15.169269 933 ts_tablet_manager.cc:1428] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 14:53:15.169394 608 heartbeater.cc:503] Master 127.30.74.126:41085 was elected leader, sending a full tablet report...
I20260810 14:53:15.169488 937 tablet_bootstrap.cc:654] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:15.169705 934 raft_consensus.cc:348] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } }
I20260810 14:53:15.169867 937 log.cc:824] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:15.169891 934 raft_consensus.cc:374] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:15.169934 934 raft_consensus.cc:729] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b6fe2f6be6a45f699c3534e102d4dcf, State: Initialized, Role: FOLLOWER
I20260810 14:53:15.170051 934 consensus_queue.cc:260] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } }
I20260810 14:53:15.170356 934 ts_tablet_manager.cc:1428] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:53:15.170665 937 tablet_bootstrap.cc:492] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f: No bootstrap required, opened a new log
I20260810 14:53:15.170754 937 ts_tablet_manager.cc:1397] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:53:15.172099 937 raft_consensus.cc:348] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } }
I20260810 14:53:15.172250 937 raft_consensus.cc:374] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:15.172349 937 raft_consensus.cc:729] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 493b4c03e00a4783aef988e4b268237f, State: Initialized, Role: FOLLOWER
I20260810 14:53:15.172395 759 heartbeater.cc:503] Master 127.30.74.126:41085 was elected leader, sending a full tablet report...
I20260810 14:53:15.172484 937 consensus_queue.cc:260] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } }
I20260810 14:53:15.172782 937 ts_tablet_manager.cc:1428] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:53:15.172799 908 heartbeater.cc:503] Master 127.30.74.126:41085 was elected leader, sending a full tablet report...
W20260810 14:53:15.202517 760 tablet.cc:2369] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:53:15.266491 938 raft_consensus.cc:482] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:15.266620 938 raft_consensus.cc:504] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } }
I20260810 14:53:15.267063 938 leader_election.cc:288] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 493b4c03e00a4783aef988e4b268237f (127.30.74.67:38345), 9b6fe2f6be6a45f699c3534e102d4dcf (127.30.74.66:34207)
I20260810 14:53:15.270049 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.267365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:43873 (local address 127.30.74.67:38345)
0810 14:53:15.267414 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:15.267416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:15.267587 (+ 171us) server_negotiation.cc:407] Connection header received
0810 14:53:15.267656 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:15.267657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:15.267668 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:15.267743 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:15.267912 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.268402 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.269322 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.269681 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.269783 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.269792 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:15.269941 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 14:53:15.269979 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 14:53:15.270116 932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.267428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:47313 (local address 127.30.74.66:34207)
0810 14:53:15.267476 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:15.267478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:15.267487 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:15.267516 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:15.267517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:15.267526 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:15.267603 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:15.267770 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.268317 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.269256 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.269634 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.269731 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.269741 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:15.270008 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 14:53:15.270049 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:53:15.270072 942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.267297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:38345 (local address 127.30.74.65:43873)
0810 14:53:15.267504 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:15.267512 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:15.267595 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:15.267746 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:15.267747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:15.267749 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:15.267839 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.267841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.268490 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.268491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.269218 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.269220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.269774 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.269775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.269858 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.269862 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:15.269921 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 14:53:15.269963 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":13,"thread_start_us":78,"threads_started":1}
I20260810 14:53:15.270301 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.267274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:34207 (local address 127.30.74.65:47313)
0810 14:53:15.267369 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:15.267382 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:15.267434 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:15.267611 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:15.267612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:15.267613 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:15.267687 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.267689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.268405 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.268406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.269140 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.269141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.269731 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.269732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.269809 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.269813 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:15.269864 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 14:53:15.269939 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":1}
I20260810 14:53:15.270677 861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87d4122ed7e94f879003d2f97164399f" candidate_uuid: "8cf6044aea834d71be137a3e7936b43d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "493b4c03e00a4783aef988e4b268237f" is_pre_election: true
I20260810 14:53:15.270675 710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87d4122ed7e94f879003d2f97164399f" candidate_uuid: "8cf6044aea834d71be137a3e7936b43d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" is_pre_election: true
I20260810 14:53:15.270807 861 raft_consensus.cc:2454] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cf6044aea834d71be137a3e7936b43d in term 0.
I20260810 14:53:15.270852 710 raft_consensus.cc:2454] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cf6044aea834d71be137a3e7936b43d in term 0.
I20260810 14:53:15.271080 490 leader_election.cc:302] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 493b4c03e00a4783aef988e4b268237f, 8cf6044aea834d71be137a3e7936b43d; no voters:
I20260810 14:53:15.271265 938 raft_consensus.cc:2781] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:15.271329 938 raft_consensus.cc:482] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:15.271368 938 raft_consensus.cc:3037] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:15.272081 938 raft_consensus.cc:504] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } }
I20260810 14:53:15.272233 938 leader_election.cc:288] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [CANDIDATE]: Term 1 election: Requested vote from peers 493b4c03e00a4783aef988e4b268237f (127.30.74.67:38345), 9b6fe2f6be6a45f699c3534e102d4dcf (127.30.74.66:34207)
I20260810 14:53:15.272423 861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87d4122ed7e94f879003d2f97164399f" candidate_uuid: "8cf6044aea834d71be137a3e7936b43d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "493b4c03e00a4783aef988e4b268237f"
I20260810 14:53:15.272500 861 raft_consensus.cc:3037] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:15.273254 861 raft_consensus.cc:2454] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cf6044aea834d71be137a3e7936b43d in term 1.
I20260810 14:53:15.273379 710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87d4122ed7e94f879003d2f97164399f" candidate_uuid: "8cf6044aea834d71be137a3e7936b43d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf"
I20260810 14:53:15.273420 490 leader_election.cc:302] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 493b4c03e00a4783aef988e4b268237f, 8cf6044aea834d71be137a3e7936b43d; no voters:
I20260810 14:53:15.273469 710 raft_consensus.cc:3037] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:15.273563 938 raft_consensus.cc:2781] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:15.273747 938 raft_consensus.cc:686] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [term 1 LEADER]: Becoming Leader. State: Replica: 8cf6044aea834d71be137a3e7936b43d, State: Running, Role: LEADER
I20260810 14:53:15.273845 938 consensus_queue.cc:237] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } }
I20260810 14:53:15.274386 710 raft_consensus.cc:2454] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cf6044aea834d71be137a3e7936b43d in term 1.
I20260810 14:53:15.274914 392 catalog_manager.cc:5495] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d reported cstate change: term changed from 0 to 1, leader changed from <none> to 8cf6044aea834d71be137a3e7936b43d (127.30.74.65). New cstate: current_term: 1 leader_uuid: "8cf6044aea834d71be137a3e7936b43d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } health_report { overall_health: HEALTHY } } }
W20260810 14:53:15.281071 609 tablet.cc:2369] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 14:53:15 dist-test-slave-0b58 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373594, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:15.328104 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.321230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:41085 (local address 127.0.0.1:41656)
0810 14:53:15.321421 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:15.321431 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:15.321522 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:15.321666 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:15.321667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:15.322002 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:15.322056 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.322059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.322747 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.322748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.323279 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.323280 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.323623 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.323624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.323676 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.324192 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:15.324197 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:15.325576 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:15.326979 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:15.326987 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:15.326989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:15.327180 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:15.327395 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:15.327396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:15.327397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:15.327456 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:15.327704 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:15.327712 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:15.327757 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 14:53:15.327835 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 14:53:15.327973 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":5,"thread_start_us":102,"threads_started":1}
I20260810 14:53:15.328111 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.321243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41656 (local address 127.30.74.126:41085)
0810 14:53:15.321322 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:15.321324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:15.321499 (+ 175us) server_negotiation.cc:407] Connection header received
0810 14:53:15.321577 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:15.321578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:15.321591 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:15.321661 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:15.322164 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.322659 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.323374 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.323553 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.323625 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.325668 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:15.325670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:15.325670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:15.325678 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:15.326882 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:15.327253 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:15.327254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:15.327254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:15.327326 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:15.327518 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:15.327519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:15.327519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:15.327633 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:15.327702 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:15.327868 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 14:53:15.328035 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":51}
I20260810 14:53:15.330235 392 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:41656:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 14:53:15.330454 392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:15.332218 678 tablet_service.cc:1467] Processing CreateTablet for tablet 3727eb2abc22474b8ef681f975e0b0f3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=157ac46d4ab8425985d38a8fc0e5d716]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:53:15.332222 539 tablet_service.cc:1467] Processing CreateTablet for tablet 3727eb2abc22474b8ef681f975e0b0f3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=157ac46d4ab8425985d38a8fc0e5d716]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:53:15.332470 539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3727eb2abc22474b8ef681f975e0b0f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:15.332475 678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3727eb2abc22474b8ef681f975e0b0f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:15.333899 933 tablet_bootstrap.cc:492] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d: Bootstrap starting.
I20260810 14:53:15.334062 838 tablet_service.cc:1467] Processing CreateTablet for tablet 3727eb2abc22474b8ef681f975e0b0f3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=157ac46d4ab8425985d38a8fc0e5d716]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:53:15.334210 838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3727eb2abc22474b8ef681f975e0b0f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:15.334316 933 tablet_bootstrap.cc:654] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:15.334952 934 tablet_bootstrap.cc:492] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf: Bootstrap starting.
I20260810 14:53:15.335301 933 tablet_bootstrap.cc:492] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d: No bootstrap required, opened a new log
I20260810 14:53:15.335336 933 ts_tablet_manager.cc:1397] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:15.335455 933 raft_consensus.cc:348] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.335505 933 raft_consensus.cc:374] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:15.335518 933 raft_consensus.cc:729] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cf6044aea834d71be137a3e7936b43d, State: Initialized, Role: FOLLOWER
I20260810 14:53:15.335587 933 consensus_queue.cc:260] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.335671 934 tablet_bootstrap.cc:654] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:15.335685 933 ts_tablet_manager.cc:1428] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:15.335723 938 tablet_replica.cc:439] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } }
I20260810 14:53:15.335791 938 tablet_replica.cc:442] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:15.335917 937 tablet_bootstrap.cc:492] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f: Bootstrap starting.
I20260810 14:53:15.336485 934 tablet_bootstrap.cc:492] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf: No bootstrap required, opened a new log
I20260810 14:53:15.336500 937 tablet_bootstrap.cc:654] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:15.336548 934 ts_tablet_manager.cc:1397] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:53:15.336716 934 raft_consensus.cc:348] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.336834 934 raft_consensus.cc:374] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:15.336862 934 raft_consensus.cc:729] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b6fe2f6be6a45f699c3534e102d4dcf, State: Initialized, Role: FOLLOWER
I20260810 14:53:15.336912 934 consensus_queue.cc:260] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.337016 934 ts_tablet_manager.cc:1428] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:15.337179 940 tablet_replica.cc:439] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } }
I20260810 14:53:15.337270 937 tablet_bootstrap.cc:492] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f: No bootstrap required, opened a new log
I20260810 14:53:15.337273 940 tablet_replica.cc:442] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:15.337311 937 ts_tablet_manager.cc:1397] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:15.337468 937 raft_consensus.cc:348] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.337548 937 raft_consensus.cc:374] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:15.337571 937 raft_consensus.cc:729] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 493b4c03e00a4783aef988e4b268237f, State: Initialized, Role: FOLLOWER
I20260810 14:53:15.337616 937 consensus_queue.cc:260] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.337718 937 ts_tablet_manager.cc:1428] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 14:53:15.338019 941 tablet_replica.cc:439] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } }
I20260810 14:53:15.338116 941 tablet_replica.cc:442] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:15.339219 940 raft_consensus.cc:482] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:15.339272 940 raft_consensus.cc:504] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.339527 940 leader_election.cc:288] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cf6044aea834d71be137a3e7936b43d (127.30.74.65:38517), 493b4c03e00a4783aef988e4b268237f (127.30.74.67:38345)
I20260810 14:53:15.342599 931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.339732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:48471 (local address 127.30.74.65:38517)
0810 14:53:15.339785 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:15.339788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:15.339836 (+ 48us) server_negotiation.cc:407] Connection header received
0810 14:53:15.339907 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:15.339908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:15.339918 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:15.339968 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:15.340183 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.340748 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.341851 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.342230 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.342322 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.342330 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:15.342491 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 14:53:15.342532 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 14:53:15.342618 930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.339732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:57751 (local address 127.30.74.67:38345)
0810 14:53:15.339798 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:15.339801 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:15.340001 (+ 200us) server_negotiation.cc:407] Connection header received
0810 14:53:15.340084 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:15.340084 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:15.340094 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:15.340134 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:15.340359 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.340836 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.341830 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.342232 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.342316 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.342324 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:15.342515 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 14:53:15.342554 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 14:53:15.342617 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.339687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:38517 (local address 127.30.74.66:48471)
0810 14:53:15.339780 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:15.339793 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:15.339852 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:15.340019 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:15.340021 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:15.340023 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:15.340118 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.340121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.340837 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.340838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.341728 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.341730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.342318 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.342319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.342417 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.342420 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:53:15.342464 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 14:53:15.342527 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":2}
I20260810 14:53:15.342626 955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.339687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:38345 (local address 127.30.74.66:57751)
0810 14:53:15.339953 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:15.339961 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:15.340004 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:15.340199 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:15.340200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:15.340201 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:15.340295 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.340297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.340919 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.340920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.341738 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.341740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.342360 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.342361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.342424 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.342427 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:53:15.342498 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 14:53:15.342534 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":23,"thread_start_us":90,"threads_started":1}
I20260810 14:53:15.342983 559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3727eb2abc22474b8ef681f975e0b0f3" candidate_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cf6044aea834d71be137a3e7936b43d" is_pre_election: true
I20260810 14:53:15.343124 559 raft_consensus.cc:2454] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b6fe2f6be6a45f699c3534e102d4dcf in term 0.
I20260810 14:53:15.343160 861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3727eb2abc22474b8ef681f975e0b0f3" candidate_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "493b4c03e00a4783aef988e4b268237f" is_pre_election: true
I20260810 14:53:15.343237 861 raft_consensus.cc:2454] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b6fe2f6be6a45f699c3534e102d4dcf in term 0.
I20260810 14:53:15.343359 637 leader_election.cc:302] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cf6044aea834d71be137a3e7936b43d, 9b6fe2f6be6a45f699c3534e102d4dcf; no voters:
I20260810 14:53:15.343487 940 raft_consensus.cc:2781] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:15.343534 940 raft_consensus.cc:482] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:15.343557 940 raft_consensus.cc:3037] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:15.344198 940 raft_consensus.cc:504] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.344365 940 leader_election.cc:288] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [CANDIDATE]: Term 1 election: Requested vote from peers 8cf6044aea834d71be137a3e7936b43d (127.30.74.65:38517), 493b4c03e00a4783aef988e4b268237f (127.30.74.67:38345)
I20260810 14:53:15.344511 559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3727eb2abc22474b8ef681f975e0b0f3" candidate_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cf6044aea834d71be137a3e7936b43d"
I20260810 14:53:15.344517 861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3727eb2abc22474b8ef681f975e0b0f3" candidate_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "493b4c03e00a4783aef988e4b268237f"
I20260810 14:53:15.344589 559 raft_consensus.cc:3037] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:15.344609 861 raft_consensus.cc:3037] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:15.345203 861 raft_consensus.cc:2454] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b6fe2f6be6a45f699c3534e102d4dcf in term 1.
I20260810 14:53:15.345230 559 raft_consensus.cc:2454] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b6fe2f6be6a45f699c3534e102d4dcf in term 1.
I20260810 14:53:15.345399 637 leader_election.cc:302] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cf6044aea834d71be137a3e7936b43d, 9b6fe2f6be6a45f699c3534e102d4dcf; no voters:
I20260810 14:53:15.345497 940 raft_consensus.cc:2781] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:15.345639 940 raft_consensus.cc:686] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 LEADER]: Becoming Leader. State: Replica: 9b6fe2f6be6a45f699c3534e102d4dcf, State: Running, Role: LEADER
I20260810 14:53:15.345773 940 consensus_queue.cc:237] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.346253 958 tablet_replica.cc:439] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf: TxnStatusTablet state changed. Reason: New leader 9b6fe2f6be6a45f699c3534e102d4dcf. Latest consensus state: current_term: 1 leader_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } }
I20260810 14:53:15.346345 958 tablet_replica.cc:442] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:53:15.346531 960 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 14:53:15.346580 390 catalog_manager.cc:5495] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b6fe2f6be6a45f699c3534e102d4dcf (127.30.74.66). New cstate: current_term: 1 leader_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } health_report { overall_health: UNKNOWN } } }
I20260810 14:53:15.346680 960 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 14:53:15.356343 932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.353535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34882 (local address 127.30.74.66:34207)
0810 14:53:15.353599 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:15.353601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:15.353614 (+ 13us) server_negotiation.cc:407] Connection header received
0810 14:53:15.353691 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:15.353692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:15.353707 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:15.353763 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:15.353890 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.354477 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.355551 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.355732 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.355786 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.355999 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:15.356089 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:15.356147 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:15.356220 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 14:53:15.356238 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 14:53:15.356374 951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.353467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:34207 (local address 127.0.0.1:34882)
0810 14:53:15.353556 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:15.353567 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:15.353611 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:15.353775 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:15.353776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:15.353778 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:53:15.353827 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.353830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.354562 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.354563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.355472 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.355475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.355824 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.355825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.355894 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.355911 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:53:15.356157 (+ 246us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:53:15.356166 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 14:53:15.356218 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 14:53:15.356295 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 14:53:15.358110 861 raft_consensus.cc:1261] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Refusing update from remote peer 9b6fe2f6be6a45f699c3534e102d4dcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:15.358112 559 raft_consensus.cc:1261] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d [term 1 FOLLOWER]: Refusing update from remote peer 9b6fe2f6be6a45f699c3534e102d4dcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:15.358443 940 consensus_queue.cc:1035] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:15.358556 958 consensus_queue.cc:1035] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:15.359683 938 tablet_replica.cc:439] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d: TxnStatusTablet state changed. Reason: New leader 9b6fe2f6be6a45f699c3534e102d4dcf. Latest consensus state: current_term: 1 leader_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } }
I20260810 14:53:15.359781 938 tablet_replica.cc:442] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:15.360087 941 tablet_replica.cc:439] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f: TxnStatusTablet state changed. Reason: New leader 9b6fe2f6be6a45f699c3534e102d4dcf. Latest consensus state: current_term: 1 leader_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } }
I20260810 14:53:15.360183 941 tablet_replica.cc:442] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:15.360772 940 tablet_replica.cc:439] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } }
I20260810 14:53:15.360858 940 tablet_replica.cc:442] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:53:15.361459 938 tablet_replica.cc:439] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } }
I20260810 14:53:15.361562 938 tablet_replica.cc:442] T 3727eb2abc22474b8ef681f975e0b0f3 P 8cf6044aea834d71be137a3e7936b43d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:15.361900 958 tablet_replica.cc:439] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } }
I20260810 14:53:15.361981 958 tablet_replica.cc:442] T 3727eb2abc22474b8ef681f975e0b0f3 P 9b6fe2f6be6a45f699c3534e102d4dcf: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:53:15.362568 941 tablet_replica.cc:439] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } } }
I20260810 14:53:15.362672 941 tablet_replica.cc:442] T 3727eb2abc22474b8ef681f975e0b0f3 P 493b4c03e00a4783aef988e4b268237f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:15.363039 965 mvcc.cc:205] Tried to move back new op lower bound from 7316986246584205312 to 7316986246536884224. Current Snapshot: MvccSnapshot[applied={T|T < 7316986246584205312 or (T in {7316986246584205312})}]
W20260810 14:53:15.364389 909 tablet.cc:2369] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:53:15.366652 931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.363573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42342 (local address 127.30.74.65:38517)
0810 14:53:15.363666 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:15.363669 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:15.363677 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:53:15.363725 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:15.363726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:15.363744 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:15.363826 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:15.363952 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.364768 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.365606 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.365922 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.365995 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.366312 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:15.366434 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:15.366522 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:15.366562 (+ 40us) server_negotiation.cc:299] Negotiation successful
0810 14:53:15.366582 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 14:53:15.372066 932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.369298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:59481 (local address 127.30.74.66:34207)
0810 14:53:15.369350 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:15.369352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:15.369461 (+ 109us) server_negotiation.cc:407] Connection header received
0810 14:53:15.369489 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:15.369490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:15.369500 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:15.369578 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:15.369678 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.370400 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.371286 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.371612 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.371680 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.371838 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:15.371874 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:15.371926 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:15.371972 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 14:53:15.371993 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 14:53:15.372085 974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.369212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:34207 (local address 127.30.74.65:59481)
0810 14:53:15.369391 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:15.369401 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:15.369451 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:15.369580 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:15.369580 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:15.369581 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:53:15.369638 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.369639 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.370496 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.370497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.371202 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.371204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.371679 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.371680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.371734 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.371748 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:53:15.371933 (+ 185us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:53:15.371934 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 14:53:15.371969 (+ 35us) client_negotiation.cc:240] Negotiation successful
0810 14:53:15.372006 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":110,"threads_started":1}
I20260810 14:53:15.374773 861 raft_consensus.cc:1261] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Refusing update from remote peer 8cf6044aea834d71be137a3e7936b43d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:15.374826 710 raft_consensus.cc:1261] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 FOLLOWER]: Refusing update from remote peer 8cf6044aea834d71be137a3e7936b43d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:15.375028 943 consensus_queue.cc:1035] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [LEADER]: Connected to new peer: Peer: permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:15.375137 938 consensus_queue.cc:1035] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:15.376812 964 mvcc.cc:205] Tried to move back new op lower bound from 7316986246653263872 to 7316986246242369536. Current Snapshot: MvccSnapshot[applied={T|T < 7316986246653263872}]
I20260810 14:53:15.376978 963 mvcc.cc:205] Tried to move back new op lower bound from 7316986246653263872 to 7316986246242369536. Current Snapshot: MvccSnapshot[applied={T|T < 7316986246653263872}]
I20260810 14:53:15.384677 931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.381919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:45065 (local address 127.30.74.65:38517)
0810 14:53:15.381965 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:15.381967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:15.381975 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:53:15.382027 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:15.382028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:15.382037 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:15.382090 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:15.382262 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.382860 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.383777 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.384079 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.384141 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.384457 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:15.384493 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:15.384542 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:15.384593 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 14:53:15.384613 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 14:53:15.384728 765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:15.381855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:38517 (local address 127.30.74.66:45065)
0810 14:53:15.381912 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:15.381924 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:15.381970 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:15.382094 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:15.382095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:15.382096 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:53:15.382165 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.382167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.382952 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.382953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.383690 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:15.383693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:15.384178 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:15.384179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:15.384287 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:15.384302 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:53:15.384543 (+ 241us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:53:15.384545 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 14:53:15.384592 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 14:53:15.384634 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260810 14:53:15.394797 392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:15.394901 392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:15.395675 392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8: Sending DeleteTablet for 3 replicas of tablet 87d4122ed7e94f879003d2f97164399f
I20260810 14:53:15.396010 838 tablet_service.cc:1514] Processing DeleteTablet for tablet 87d4122ed7e94f879003d2f97164399f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:15 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:54136
I20260810 14:53:15.396062 678 tablet_service.cc:1514] Processing DeleteTablet for tablet 87d4122ed7e94f879003d2f97164399f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:15 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:34872
I20260810 14:53:15.396103 539 tablet_service.cc:1514] Processing DeleteTablet for tablet 87d4122ed7e94f879003d2f97164399f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:15 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:42336
I20260810 14:53:15.396649 392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:53:15.396801 392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:15.398173 838 tablet_service.cc:1467] Processing CreateTablet for tablet 91a06796ba0a42f3bee1a26ae8922f05 (DEFAULT_TABLE table=test-table [id=cb82bc544580408aae6b6a9d86e9b9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:15.398188 539 tablet_service.cc:1467] Processing CreateTablet for tablet 91a06796ba0a42f3bee1a26ae8922f05 (DEFAULT_TABLE table=test-table [id=cb82bc544580408aae6b6a9d86e9b9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:15.398334 539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91a06796ba0a42f3bee1a26ae8922f05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:15.398355 838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91a06796ba0a42f3bee1a26ae8922f05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:15.399675 933 tablet_bootstrap.cc:492] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d: Bootstrap starting.
I20260810 14:53:15.399727 937 tablet_bootstrap.cc:492] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f: Bootstrap starting.
I20260810 14:53:15.399995 678 tablet_service.cc:1467] Processing CreateTablet for tablet 91a06796ba0a42f3bee1a26ae8922f05 (DEFAULT_TABLE table=test-table [id=cb82bc544580408aae6b6a9d86e9b9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:15.400115 678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91a06796ba0a42f3bee1a26ae8922f05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:15.400156 937 tablet_bootstrap.cc:654] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:15.400156 933 tablet_bootstrap.cc:654] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:15.401126 933 tablet_bootstrap.cc:492] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d: No bootstrap required, opened a new log
I20260810 14:53:15.401170 933 ts_tablet_manager.cc:1397] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 14:53:15.401309 933 raft_consensus.cc:348] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.401378 933 raft_consensus.cc:374] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:15.401399 933 raft_consensus.cc:729] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cf6044aea834d71be137a3e7936b43d, State: Initialized, Role: FOLLOWER
I20260810 14:53:15.401438 934 tablet_bootstrap.cc:492] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf: Bootstrap starting.
I20260810 14:53:15.401443 933 consensus_queue.cc:260] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.401535 933 ts_tablet_manager.cc:1428] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 14:53:15.401880 934 tablet_bootstrap.cc:654] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:15.402493 937 tablet_bootstrap.cc:492] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f: No bootstrap required, opened a new log
I20260810 14:53:15.402544 937 ts_tablet_manager.cc:1397] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260810 14:53:15.402683 937 raft_consensus.cc:348] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.402786 937 raft_consensus.cc:374] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:15.402814 937 raft_consensus.cc:729] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 493b4c03e00a4783aef988e4b268237f, State: Initialized, Role: FOLLOWER
I20260810 14:53:15.402853 937 consensus_queue.cc:260] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.402953 937 ts_tablet_manager.cc:1428] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:15.403259 934 tablet_bootstrap.cc:492] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf: No bootstrap required, opened a new log
I20260810 14:53:15.403311 934 ts_tablet_manager.cc:1397] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:53:15.403442 934 raft_consensus.cc:348] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.403503 934 raft_consensus.cc:374] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:15.403522 934 raft_consensus.cc:729] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b6fe2f6be6a45f699c3534e102d4dcf, State: Initialized, Role: FOLLOWER
I20260810 14:53:15.403563 934 consensus_queue.cc:260] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.403642 934 ts_tablet_manager.cc:1428] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:15.404152 987 tablet_replica.cc:330] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d: stopping tablet replica
I20260810 14:53:15.404366 984 tablet_replica.cc:330] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f: stopping tablet replica
I20260810 14:53:15.404502 984 raft_consensus.cc:2229] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:15.404573 984 raft_consensus.cc:2258] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:15.404843 987 raft_consensus.cc:2229] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:53:15.405009 987 raft_consensus.cc:2258] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:15.405282 985 tablet_replica.cc:330] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf: stopping tablet replica
I20260810 14:53:15.405428 985 raft_consensus.cc:2229] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:15.405530 985 raft_consensus.cc:2258] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:15.405575 987 ts_tablet_manager.cc:1905] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:15.406044 985 ts_tablet_manager.cc:1905] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:15.407557 987 ts_tablet_manager.cc:1918] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:53:15.407637 987 log.cc:1196] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/wal/wals/87d4122ed7e94f879003d2f97164399f
I20260810 14:53:15.407917 987 ts_tablet_manager.cc:1939] T 87d4122ed7e94f879003d2f97164399f P 8cf6044aea834d71be137a3e7936b43d: Deleting consensus metadata
I20260810 14:53:15.408342 376 catalog_manager.cc:4841] TS 8cf6044aea834d71be137a3e7936b43d (127.30.74.65:38517): tablet 87d4122ed7e94f879003d2f97164399f (table test-table [id=dde1538ebfbf4c259840b54fb1dae8ff]) successfully deleted
I20260810 14:53:15.408370 985 ts_tablet_manager.cc:1918] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:53:15.408529 985 log.cc:1196] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/87d4122ed7e94f879003d2f97164399f
I20260810 14:53:15.408879 985 ts_tablet_manager.cc:1939] T 87d4122ed7e94f879003d2f97164399f P 9b6fe2f6be6a45f699c3534e102d4dcf: Deleting consensus metadata
I20260810 14:53:15.409335 377 catalog_manager.cc:4841] TS 9b6fe2f6be6a45f699c3534e102d4dcf (127.30.74.66:34207): tablet 87d4122ed7e94f879003d2f97164399f (table test-table [id=dde1538ebfbf4c259840b54fb1dae8ff]) successfully deleted
I20260810 14:53:15.412734 984 ts_tablet_manager.cc:1905] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:15.414482 984 ts_tablet_manager.cc:1918] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:53:15.414568 984 log.cc:1196] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/87d4122ed7e94f879003d2f97164399f
I20260810 14:53:15.414853 984 ts_tablet_manager.cc:1939] T 87d4122ed7e94f879003d2f97164399f P 493b4c03e00a4783aef988e4b268237f: Deleting consensus metadata
I20260810 14:53:15.415869 378 catalog_manager.cc:4841] TS 493b4c03e00a4783aef988e4b268237f (127.30.74.67:38345): tablet 87d4122ed7e94f879003d2f97164399f (table test-table [id=dde1538ebfbf4c259840b54fb1dae8ff]) successfully deleted
I20260810 14:53:15.430707 943 raft_consensus.cc:482] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:15.430837 943 raft_consensus.cc:504] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.431020 943 leader_election.cc:288] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b6fe2f6be6a45f699c3534e102d4dcf (127.30.74.66:34207), 493b4c03e00a4783aef988e4b268237f (127.30.74.67:38345)
I20260810 14:53:15.431181 710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91a06796ba0a42f3bee1a26ae8922f05" candidate_uuid: "8cf6044aea834d71be137a3e7936b43d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" is_pre_election: true
I20260810 14:53:15.431293 710 raft_consensus.cc:2454] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cf6044aea834d71be137a3e7936b43d in term 0.
I20260810 14:53:15.431377 861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91a06796ba0a42f3bee1a26ae8922f05" candidate_uuid: "8cf6044aea834d71be137a3e7936b43d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "493b4c03e00a4783aef988e4b268237f" is_pre_election: true
I20260810 14:53:15.431465 861 raft_consensus.cc:2454] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cf6044aea834d71be137a3e7936b43d in term 0.
I20260810 14:53:15.431862 489 leader_election.cc:302] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cf6044aea834d71be137a3e7936b43d, 9b6fe2f6be6a45f699c3534e102d4dcf; no voters:
I20260810 14:53:15.432019 943 raft_consensus.cc:2781] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:15.432076 943 raft_consensus.cc:482] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:15.432101 943 raft_consensus.cc:3037] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:15.433091 943 raft_consensus.cc:504] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.433243 943 leader_election.cc:288] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [CANDIDATE]: Term 1 election: Requested vote from peers 9b6fe2f6be6a45f699c3534e102d4dcf (127.30.74.66:34207), 493b4c03e00a4783aef988e4b268237f (127.30.74.67:38345)
I20260810 14:53:15.433468 861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91a06796ba0a42f3bee1a26ae8922f05" candidate_uuid: "8cf6044aea834d71be137a3e7936b43d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "493b4c03e00a4783aef988e4b268237f"
I20260810 14:53:15.433475 710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91a06796ba0a42f3bee1a26ae8922f05" candidate_uuid: "8cf6044aea834d71be137a3e7936b43d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf"
I20260810 14:53:15.433543 861 raft_consensus.cc:3037] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:15.433547 710 raft_consensus.cc:3037] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:15.434279 710 raft_consensus.cc:2454] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cf6044aea834d71be137a3e7936b43d in term 1.
I20260810 14:53:15.434319 861 raft_consensus.cc:2454] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cf6044aea834d71be137a3e7936b43d in term 1.
I20260810 14:53:15.434444 489 leader_election.cc:302] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cf6044aea834d71be137a3e7936b43d, 9b6fe2f6be6a45f699c3534e102d4dcf; no voters:
I20260810 14:53:15.434526 943 raft_consensus.cc:2781] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:15.434562 943 raft_consensus.cc:686] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [term 1 LEADER]: Becoming Leader. State: Replica: 8cf6044aea834d71be137a3e7936b43d, State: Running, Role: LEADER
I20260810 14:53:15.434603 943 consensus_queue.cc:237] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:15.435252 390 catalog_manager.cc:5495] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d reported cstate change: term changed from 0 to 1, leader changed from <none> to 8cf6044aea834d71be137a3e7936b43d (127.30.74.65). New cstate: current_term: 1 leader_uuid: "8cf6044aea834d71be137a3e7936b43d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } health_report { overall_health: UNKNOWN } } }
I20260810 14:53:15.443075 861 raft_consensus.cc:1261] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Refusing update from remote peer 8cf6044aea834d71be137a3e7936b43d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:15.443118 710 raft_consensus.cc:1261] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 FOLLOWER]: Refusing update from remote peer 8cf6044aea834d71be137a3e7936b43d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:15.443276 980 consensus_queue.cc:1035] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [LEADER]: Connected to new peer: Peer: permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:15.443362 943 consensus_queue.cc:1035] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 14:53:15.449826 964 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 14:53:16.970414 390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:16.970520 390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:16.971338 390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8: Sending DeleteTablet for 3 replicas of tablet 91a06796ba0a42f3bee1a26ae8922f05
I20260810 14:53:16.971608 539 tablet_service.cc:1514] Processing DeleteTablet for tablet 91a06796ba0a42f3bee1a26ae8922f05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:16 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:42336
I20260810 14:53:16.971637 838 tablet_service.cc:1514] Processing DeleteTablet for tablet 91a06796ba0a42f3bee1a26ae8922f05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:16 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:54136
I20260810 14:53:16.971926 1020 tablet_replica.cc:330] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f: stopping tablet replica
I20260810 14:53:16.971916 678 tablet_service.cc:1514] Processing DeleteTablet for tablet 91a06796ba0a42f3bee1a26ae8922f05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:16 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:34872
I20260810 14:53:16.971949 1019 tablet_replica.cc:330] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d: stopping tablet replica
I20260810 14:53:16.972007 1019 raft_consensus.cc:2229] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:53:16.972069 1020 raft_consensus.cc:2229] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:16.972096 1021 tablet_replica.cc:330] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf: stopping tablet replica
I20260810 14:53:16.972146 1021 raft_consensus.cc:2229] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:16.972149 1020 raft_consensus.cc:2258] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:16.972110 1019 raft_consensus.cc:2258] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:16.972422 1019 ts_tablet_manager.cc:1905] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:16.972415 1021 raft_consensus.cc:2258] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:16.972750 1021 ts_tablet_manager.cc:1905] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:16.974112 1019 ts_tablet_manager.cc:1918] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:53:16.974380 1020 ts_tablet_manager.cc:1905] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:16.974726 1021 ts_tablet_manager.cc:1918] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:53:16.974867 1021 log.cc:1196] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/91a06796ba0a42f3bee1a26ae8922f05
I20260810 14:53:16.975188 1021 ts_tablet_manager.cc:1939] T 91a06796ba0a42f3bee1a26ae8922f05 P 9b6fe2f6be6a45f699c3534e102d4dcf: Deleting consensus metadata
I20260810 14:53:16.975581 377 catalog_manager.cc:4841] TS 9b6fe2f6be6a45f699c3534e102d4dcf (127.30.74.66:34207): tablet 91a06796ba0a42f3bee1a26ae8922f05 (table test-table [id=cb82bc544580408aae6b6a9d86e9b9b1]) successfully deleted
I20260810 14:53:16.975934 1020 ts_tablet_manager.cc:1918] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:53:16.975986 1020 log.cc:1196] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/91a06796ba0a42f3bee1a26ae8922f05
I20260810 14:53:16.976200 1020 ts_tablet_manager.cc:1939] T 91a06796ba0a42f3bee1a26ae8922f05 P 493b4c03e00a4783aef988e4b268237f: Deleting consensus metadata
I20260810 14:53:16.976553 378 catalog_manager.cc:4841] TS 493b4c03e00a4783aef988e4b268237f (127.30.74.67:38345): tablet 91a06796ba0a42f3bee1a26ae8922f05 (table test-table [id=cb82bc544580408aae6b6a9d86e9b9b1]) successfully deleted
I20260810 14:53:16.978847 1019 log.cc:1196] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/wal/wals/91a06796ba0a42f3bee1a26ae8922f05
I20260810 14:53:16.979149 1019 ts_tablet_manager.cc:1939] T 91a06796ba0a42f3bee1a26ae8922f05 P 8cf6044aea834d71be137a3e7936b43d: Deleting consensus metadata
I20260810 14:53:16.979494 376 catalog_manager.cc:4841] TS 8cf6044aea834d71be137a3e7936b43d (127.30.74.65:38517): tablet 91a06796ba0a42f3bee1a26ae8922f05 (table test-table [id=cb82bc544580408aae6b6a9d86e9b9b1]) successfully deleted
Aug 10 14:53:16 dist-test-slave-0b58 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373596, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 14:53:16 dist-test-slave-0b58 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373596, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:16.998327 1030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:16.990852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41666 (local address 127.30.74.126:41085)
0810 14:53:16.991010 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:16.991012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:16.991024 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:53:16.991095 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:16.991096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:16.991106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:16.991178 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:16.991723 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:16.992240 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:16.993081 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:16.993296 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:16.993363 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:16.995653 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:16.995655 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:16.995655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:16.995662 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:16.997267 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:16.997617 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:16.997618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:16.997618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:16.997682 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:16.997875 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:16.997876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:16.997876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:16.997991 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:16.998069 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:16.998177 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 14:53:16.998247 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":100,"threads_started":1}
I20260810 14:53:16.999974 390 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:53:17.000162 390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:17.001611 539 tablet_service.cc:1467] Processing CreateTablet for tablet feec99bb7b824a5c924176fdfd3112a3 (DEFAULT_TABLE table=test-table [id=f45a0a115a2d406e952ac6097efe51b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:17.001648 838 tablet_service.cc:1467] Processing CreateTablet for tablet feec99bb7b824a5c924176fdfd3112a3 (DEFAULT_TABLE table=test-table [id=f45a0a115a2d406e952ac6097efe51b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:17.001641 678 tablet_service.cc:1467] Processing CreateTablet for tablet feec99bb7b824a5c924176fdfd3112a3 (DEFAULT_TABLE table=test-table [id=f45a0a115a2d406e952ac6097efe51b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:17.001767 539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feec99bb7b824a5c924176fdfd3112a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:17.001773 838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feec99bb7b824a5c924176fdfd3112a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:17.001814 678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feec99bb7b824a5c924176fdfd3112a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:17.003077 1031 tablet_bootstrap.cc:492] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf: Bootstrap starting.
I20260810 14:53:17.003159 1033 tablet_bootstrap.cc:492] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d: Bootstrap starting.
I20260810 14:53:17.003536 1033 tablet_bootstrap.cc:654] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:17.003536 1031 tablet_bootstrap.cc:654] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:17.003875 1032 tablet_bootstrap.cc:492] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f: Bootstrap starting.
I20260810 14:53:17.004200 1033 tablet_bootstrap.cc:492] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d: No bootstrap required, opened a new log
I20260810 14:53:17.004359 1032 tablet_bootstrap.cc:654] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:17.004359 1033 ts_tablet_manager.cc:1397] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:17.004416 1031 tablet_bootstrap.cc:492] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf: No bootstrap required, opened a new log
I20260810 14:53:17.004494 1031 ts_tablet_manager.cc:1397] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:17.004498 1033 raft_consensus.cc:348] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.004590 1033 raft_consensus.cc:374] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:17.004635 1031 raft_consensus.cc:348] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.004719 1033 raft_consensus.cc:729] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cf6044aea834d71be137a3e7936b43d, State: Initialized, Role: FOLLOWER
I20260810 14:53:17.004771 1031 raft_consensus.cc:374] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:17.004830 1031 raft_consensus.cc:729] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b6fe2f6be6a45f699c3534e102d4dcf, State: Initialized, Role: FOLLOWER
I20260810 14:53:17.004912 1031 consensus_queue.cc:260] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.004901 1033 consensus_queue.cc:260] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.005100 1031 ts_tablet_manager.cc:1428] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 14:53:17.005273 1033 ts_tablet_manager.cc:1428] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:17.005285 1032 tablet_bootstrap.cc:492] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f: No bootstrap required, opened a new log
I20260810 14:53:17.005360 1032 ts_tablet_manager.cc:1397] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 14:53:17.005474 1032 raft_consensus.cc:348] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.005620 1032 raft_consensus.cc:374] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:17.005690 1032 raft_consensus.cc:729] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 493b4c03e00a4783aef988e4b268237f, State: Initialized, Role: FOLLOWER
I20260810 14:53:17.005754 1032 consensus_queue.cc:260] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.005932 1032 ts_tablet_manager.cc:1428] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:17.299186 994 raft_consensus.cc:482] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:17.299321 994 raft_consensus.cc:504] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.299539 994 leader_election.cc:288] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cf6044aea834d71be137a3e7936b43d (127.30.74.65:38517), 493b4c03e00a4783aef988e4b268237f (127.30.74.67:38345)
I20260810 14:53:17.299892 559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feec99bb7b824a5c924176fdfd3112a3" candidate_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cf6044aea834d71be137a3e7936b43d" is_pre_election: true
I20260810 14:53:17.299894 860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feec99bb7b824a5c924176fdfd3112a3" candidate_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "493b4c03e00a4783aef988e4b268237f" is_pre_election: true
I20260810 14:53:17.300014 559 raft_consensus.cc:2454] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b6fe2f6be6a45f699c3534e102d4dcf in term 0.
I20260810 14:53:17.300010 860 raft_consensus.cc:2454] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b6fe2f6be6a45f699c3534e102d4dcf in term 0.
I20260810 14:53:17.300238 639 leader_election.cc:302] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 493b4c03e00a4783aef988e4b268237f, 9b6fe2f6be6a45f699c3534e102d4dcf; no voters:
I20260810 14:53:17.300457 994 raft_consensus.cc:2781] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:17.300524 994 raft_consensus.cc:482] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:17.300550 994 raft_consensus.cc:3037] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:17.301434 994 raft_consensus.cc:504] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.301616 994 leader_election.cc:288] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [CANDIDATE]: Term 1 election: Requested vote from peers 8cf6044aea834d71be137a3e7936b43d (127.30.74.65:38517), 493b4c03e00a4783aef988e4b268237f (127.30.74.67:38345)
I20260810 14:53:17.301815 860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feec99bb7b824a5c924176fdfd3112a3" candidate_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "493b4c03e00a4783aef988e4b268237f"
I20260810 14:53:17.301822 559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feec99bb7b824a5c924176fdfd3112a3" candidate_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cf6044aea834d71be137a3e7936b43d"
I20260810 14:53:17.301926 559 raft_consensus.cc:3037] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:17.301929 860 raft_consensus.cc:3037] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:17.302634 860 raft_consensus.cc:2454] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b6fe2f6be6a45f699c3534e102d4dcf in term 1.
I20260810 14:53:17.302625 559 raft_consensus.cc:2454] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b6fe2f6be6a45f699c3534e102d4dcf in term 1.
I20260810 14:53:17.302865 637 leader_election.cc:302] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cf6044aea834d71be137a3e7936b43d, 9b6fe2f6be6a45f699c3534e102d4dcf; no voters:
I20260810 14:53:17.302999 994 raft_consensus.cc:2781] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:17.303069 994 raft_consensus.cc:686] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 LEADER]: Becoming Leader. State: Replica: 9b6fe2f6be6a45f699c3534e102d4dcf, State: Running, Role: LEADER
I20260810 14:53:17.303126 994 consensus_queue.cc:237] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.303848 392 catalog_manager.cc:5495] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b6fe2f6be6a45f699c3534e102d4dcf (127.30.74.66). New cstate: current_term: 1 leader_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } health_report { overall_health: UNKNOWN } } }
I20260810 14:53:17.363176 1041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:17.360644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34896 (local address 127.30.74.66:34207)
0810 14:53:17.360814 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:17.360816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:17.360855 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:53:17.360966 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:17.360967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:17.360981 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:17.361053 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:17.361182 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:17.361668 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:17.362573 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:17.362779 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:17.362835 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:17.362944 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:17.362971 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:17.363004 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:17.363053 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 14:53:17.363070 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
I20260810 14:53:17.368148 1044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:17.365756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:34207 (local address 127.30.74.66:38769)
0810 14:53:17.365940 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:17.365951 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:17.366033 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:17.366185 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:17.366186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:17.366188 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:53:17.366250 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:17.366252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:17.366828 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:17.366829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:17.367383 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:17.367384 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:17.367735 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:17.367736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:17.367793 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:17.367797 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:53:17.367985 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:53:17.367987 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 14:53:17.368030 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 14:53:17.368072 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":2,"thread_start_us":113,"threads_started":1}
I20260810 14:53:17.368148 1041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:17.365834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:38769 (local address 127.30.74.66:34207)
0810 14:53:17.365914 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:17.365916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:17.366023 (+ 107us) server_negotiation.cc:407] Connection header received
0810 14:53:17.366102 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:17.366102 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:17.366115 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:17.366180 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:17.366305 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:17.366745 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:17.367472 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:17.367660 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:17.367735 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:17.367877 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:17.367912 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:17.367969 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:17.368033 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 14:53:17.368058 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":55}
I20260810 14:53:17.370535 860 raft_consensus.cc:1261] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Refusing update from remote peer 9b6fe2f6be6a45f699c3534e102d4dcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:17.370563 559 raft_consensus.cc:1261] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d [term 1 FOLLOWER]: Refusing update from remote peer 9b6fe2f6be6a45f699c3534e102d4dcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:17.370808 1002 consensus_queue.cc:1035] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:17.370924 994 consensus_queue.cc:1035] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:17.382270 390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:17.382383 390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:17.383133 390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8: Sending DeleteTablet for 3 replicas of tablet feec99bb7b824a5c924176fdfd3112a3
I20260810 14:53:17.383414 539 tablet_service.cc:1514] Processing DeleteTablet for tablet feec99bb7b824a5c924176fdfd3112a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:17 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:42336
I20260810 14:53:17.383526 1019 tablet_replica.cc:330] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d: stopping tablet replica
I20260810 14:53:17.383580 1019 raft_consensus.cc:2229] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:17.383617 1019 raft_consensus.cc:2258] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:17.383594 838 tablet_service.cc:1514] Processing DeleteTablet for tablet feec99bb7b824a5c924176fdfd3112a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:17 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:54136
I20260810 14:53:17.383698 689 tablet_service.cc:1514] Processing DeleteTablet for tablet feec99bb7b824a5c924176fdfd3112a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:17 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:34872
I20260810 14:53:17.383713 1020 tablet_replica.cc:330] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f: stopping tablet replica
I20260810 14:53:17.383790 1020 raft_consensus.cc:2229] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:17.383810 1021 tablet_replica.cc:330] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf: stopping tablet replica
I20260810 14:53:17.383839 1020 raft_consensus.cc:2258] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:17.383859 1021 raft_consensus.cc:2229] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:53:17.383975 1021 raft_consensus.cc:2258] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:17.384102 390 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:53:17.384306 390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:17.384357 1019 ts_tablet_manager.cc:1905] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:17.384706 1020 ts_tablet_manager.cc:1905] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:17.386372 1019 ts_tablet_manager.cc:1918] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:53:17.386384 1021 ts_tablet_manager.cc:1905] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:17.386400 1020 ts_tablet_manager.cc:1918] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:53:17.386454 1020 log.cc:1196] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/feec99bb7b824a5c924176fdfd3112a3
I20260810 14:53:17.386461 1019 log.cc:1196] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-0/wal/wals/feec99bb7b824a5c924176fdfd3112a3
I20260810 14:53:17.386684 1020 ts_tablet_manager.cc:1939] T feec99bb7b824a5c924176fdfd3112a3 P 493b4c03e00a4783aef988e4b268237f: Deleting consensus metadata
I20260810 14:53:17.386729 1019 ts_tablet_manager.cc:1939] T feec99bb7b824a5c924176fdfd3112a3 P 8cf6044aea834d71be137a3e7936b43d: Deleting consensus metadata
I20260810 14:53:17.387189 378 catalog_manager.cc:4841] TS 493b4c03e00a4783aef988e4b268237f (127.30.74.67:38345): tablet feec99bb7b824a5c924176fdfd3112a3 (table test-table [id=f45a0a115a2d406e952ac6097efe51b1]) successfully deleted
I20260810 14:53:17.388007 1021 ts_tablet_manager.cc:1918] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:53:17.388074 1021 log.cc:1196] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/feec99bb7b824a5c924176fdfd3112a3
I20260810 14:53:17.388069 376 catalog_manager.cc:4841] TS 8cf6044aea834d71be137a3e7936b43d (127.30.74.65:38517): tablet feec99bb7b824a5c924176fdfd3112a3 (table test-table [id=f45a0a115a2d406e952ac6097efe51b1]) successfully deleted
I20260810 14:53:17.388346 1021 ts_tablet_manager.cc:1939] T feec99bb7b824a5c924176fdfd3112a3 P 9b6fe2f6be6a45f699c3534e102d4dcf: Deleting consensus metadata
I20260810 14:53:17.388674 377 catalog_manager.cc:4841] TS 9b6fe2f6be6a45f699c3534e102d4dcf (127.30.74.66:34207): tablet feec99bb7b824a5c924176fdfd3112a3 (table test-table [id=f45a0a115a2d406e952ac6097efe51b1]) successfully deleted
I20260810 14:53:17.389635 689 tablet_service.cc:1467] Processing CreateTablet for tablet 5a07ea2a24df48a49c9432288dcb089e (DEFAULT_TABLE table=test-table [id=8a5c17212b9743d78f71d0fa1847e966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:17.389720 539 tablet_service.cc:1467] Processing CreateTablet for tablet 5a07ea2a24df48a49c9432288dcb089e (DEFAULT_TABLE table=test-table [id=8a5c17212b9743d78f71d0fa1847e966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:17.389775 689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a07ea2a24df48a49c9432288dcb089e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:17.389827 539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a07ea2a24df48a49c9432288dcb089e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:17.391083 1033 tablet_bootstrap.cc:492] T 5a07ea2a24df48a49c9432288dcb089e P 8cf6044aea834d71be137a3e7936b43d: Bootstrap starting.
I20260810 14:53:17.391161 1031 tablet_bootstrap.cc:492] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf: Bootstrap starting.
I20260810 14:53:17.391448 1033 tablet_bootstrap.cc:654] T 5a07ea2a24df48a49c9432288dcb089e P 8cf6044aea834d71be137a3e7936b43d: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:17.391465 1031 tablet_bootstrap.cc:654] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:17.391737 838 tablet_service.cc:1467] Processing CreateTablet for tablet 5a07ea2a24df48a49c9432288dcb089e (DEFAULT_TABLE table=test-table [id=8a5c17212b9743d78f71d0fa1847e966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:17.391860 838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a07ea2a24df48a49c9432288dcb089e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:17.392037 1031 tablet_bootstrap.cc:492] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf: No bootstrap required, opened a new log
I20260810 14:53:17.392086 1031 ts_tablet_manager.cc:1397] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:17.392194 1031 raft_consensus.cc:348] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.392292 1031 raft_consensus.cc:374] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:17.392314 1031 raft_consensus.cc:729] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b6fe2f6be6a45f699c3534e102d4dcf, State: Initialized, Role: FOLLOWER
I20260810 14:53:17.392362 1031 consensus_queue.cc:260] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.392455 1031 ts_tablet_manager.cc:1428] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:17.392905 1032 tablet_bootstrap.cc:492] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f: Bootstrap starting.
I20260810 14:53:17.393301 1032 tablet_bootstrap.cc:654] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:17.394099 1032 tablet_bootstrap.cc:492] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f: No bootstrap required, opened a new log
I20260810 14:53:17.394166 1032 ts_tablet_manager.cc:1397] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:17.394330 1032 raft_consensus.cc:348] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.394408 1032 raft_consensus.cc:374] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:17.394433 1032 raft_consensus.cc:729] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 493b4c03e00a4783aef988e4b268237f, State: Initialized, Role: FOLLOWER
I20260810 14:53:17.394477 1032 consensus_queue.cc:260] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.394605 1032 ts_tablet_manager.cc:1428] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:17.396739 1033 tablet_bootstrap.cc:492] T 5a07ea2a24df48a49c9432288dcb089e P 8cf6044aea834d71be137a3e7936b43d: No bootstrap required, opened a new log
I20260810 14:53:17.396811 1033 ts_tablet_manager.cc:1397] T 5a07ea2a24df48a49c9432288dcb089e P 8cf6044aea834d71be137a3e7936b43d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260810 14:53:17.396940 1033 raft_consensus.cc:348] T 5a07ea2a24df48a49c9432288dcb089e P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.397022 1033 raft_consensus.cc:374] T 5a07ea2a24df48a49c9432288dcb089e P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:17.397042 1033 raft_consensus.cc:729] T 5a07ea2a24df48a49c9432288dcb089e P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cf6044aea834d71be137a3e7936b43d, State: Initialized, Role: FOLLOWER
I20260810 14:53:17.397089 1033 consensus_queue.cc:260] T 5a07ea2a24df48a49c9432288dcb089e P 8cf6044aea834d71be137a3e7936b43d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.397189 1033 ts_tablet_manager.cc:1428] T 5a07ea2a24df48a49c9432288dcb089e P 8cf6044aea834d71be137a3e7936b43d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:17.483176 997 raft_consensus.cc:482] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:17.483299 997 raft_consensus.cc:504] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.483496 997 leader_election.cc:288] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cf6044aea834d71be137a3e7936b43d (127.30.74.65:38517), 493b4c03e00a4783aef988e4b268237f (127.30.74.67:38345)
I20260810 14:53:17.483747 559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a07ea2a24df48a49c9432288dcb089e" candidate_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cf6044aea834d71be137a3e7936b43d" is_pre_election: true
I20260810 14:53:17.483778 860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a07ea2a24df48a49c9432288dcb089e" candidate_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "493b4c03e00a4783aef988e4b268237f" is_pre_election: true
I20260810 14:53:17.483880 559 raft_consensus.cc:2454] T 5a07ea2a24df48a49c9432288dcb089e P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b6fe2f6be6a45f699c3534e102d4dcf in term 0.
I20260810 14:53:17.483901 860 raft_consensus.cc:2454] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b6fe2f6be6a45f699c3534e102d4dcf in term 0.
I20260810 14:53:17.484068 637 leader_election.cc:302] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cf6044aea834d71be137a3e7936b43d, 9b6fe2f6be6a45f699c3534e102d4dcf; no voters:
I20260810 14:53:17.484213 997 raft_consensus.cc:2781] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:17.484315 997 raft_consensus.cc:482] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:17.484349 997 raft_consensus.cc:3037] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:17.485195 997 raft_consensus.cc:504] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.485366 997 leader_election.cc:288] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [CANDIDATE]: Term 1 election: Requested vote from peers 8cf6044aea834d71be137a3e7936b43d (127.30.74.65:38517), 493b4c03e00a4783aef988e4b268237f (127.30.74.67:38345)
I20260810 14:53:17.485505 860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a07ea2a24df48a49c9432288dcb089e" candidate_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "493b4c03e00a4783aef988e4b268237f"
I20260810 14:53:17.485582 860 raft_consensus.cc:3037] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:17.485554 559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a07ea2a24df48a49c9432288dcb089e" candidate_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cf6044aea834d71be137a3e7936b43d"
I20260810 14:53:17.485625 559 raft_consensus.cc:3037] T 5a07ea2a24df48a49c9432288dcb089e P 8cf6044aea834d71be137a3e7936b43d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:17.486383 860 raft_consensus.cc:2454] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b6fe2f6be6a45f699c3534e102d4dcf in term 1.
I20260810 14:53:17.486383 559 raft_consensus.cc:2454] T 5a07ea2a24df48a49c9432288dcb089e P 8cf6044aea834d71be137a3e7936b43d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b6fe2f6be6a45f699c3534e102d4dcf in term 1.
I20260810 14:53:17.486569 639 leader_election.cc:302] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 493b4c03e00a4783aef988e4b268237f, 9b6fe2f6be6a45f699c3534e102d4dcf; no voters:
I20260810 14:53:17.486672 997 raft_consensus.cc:2781] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:17.486716 997 raft_consensus.cc:686] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 LEADER]: Becoming Leader. State: Replica: 9b6fe2f6be6a45f699c3534e102d4dcf, State: Running, Role: LEADER
I20260810 14:53:17.486770 997 consensus_queue.cc:237] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } }
I20260810 14:53:17.487381 390 catalog_manager.cc:5495] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b6fe2f6be6a45f699c3534e102d4dcf (127.30.74.66). New cstate: current_term: 1 leader_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b6fe2f6be6a45f699c3534e102d4dcf" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 } health_report { overall_health: UNKNOWN } } }
I20260810 14:53:17.515242 860 raft_consensus.cc:1261] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Refusing update from remote peer 9b6fe2f6be6a45f699c3534e102d4dcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:17.515260 559 raft_consensus.cc:1261] T 5a07ea2a24df48a49c9432288dcb089e P 8cf6044aea834d71be137a3e7936b43d [term 1 FOLLOWER]: Refusing update from remote peer 9b6fe2f6be6a45f699c3534e102d4dcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:17.515537 1002 consensus_queue.cc:1035] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "493b4c03e00a4783aef988e4b268237f" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:17.515656 994 consensus_queue.cc:1035] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cf6044aea834d71be137a3e7936b43d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 14:53:17.520601 995 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 14:53:19.040959 390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:19.041060 390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:19.041906 390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9085c84712ab4adba9211ee9b47216f8: Sending DeleteTablet for 3 replicas of tablet 5a07ea2a24df48a49c9432288dcb089e
I20260810 14:53:19.042171 678 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a07ea2a24df48a49c9432288dcb089e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:19 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:34872
I20260810 14:53:19.042218 539 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a07ea2a24df48a49c9432288dcb089e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:19 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:42336
I20260810 14:53:19.042225 838 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a07ea2a24df48a49c9432288dcb089e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:19 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:54136
I20260810 14:53:19.042395 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 470
I20260810 14:53:19.042418 1077 tablet_replica.cc:330] T 5a07ea2a24df48a49c9432288dcb089e P 8cf6044aea834d71be137a3e7936b43d: stopping tablet replica
I20260810 14:53:19.045101 1075 tablet_replica.cc:330] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf: stopping tablet replica
I20260810 14:53:19.045162 1075 raft_consensus.cc:2229] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:53:19.045235 1075 raft_consensus.cc:2258] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:19.045398 1076 tablet_replica.cc:330] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f: stopping tablet replica
I20260810 14:53:19.045463 1076 raft_consensus.cc:2229] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:19.045508 1076 raft_consensus.cc:2258] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:19.045562 1075 ts_tablet_manager.cc:1905] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:19.045773 1076 ts_tablet_manager.cc:1905] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:19.047508 1076 ts_tablet_manager.cc:1918] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:53:19.047508 1075 ts_tablet_manager.cc:1918] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:53:19.047574 1076 log.cc:1196] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/5a07ea2a24df48a49c9432288dcb089e
I20260810 14:53:19.047574 1075 log.cc:1196] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/5a07ea2a24df48a49c9432288dcb089e
I20260810 14:53:19.047806 1076 ts_tablet_manager.cc:1939] T 5a07ea2a24df48a49c9432288dcb089e P 493b4c03e00a4783aef988e4b268237f: Deleting consensus metadata
I20260810 14:53:19.047863 1075 ts_tablet_manager.cc:1939] T 5a07ea2a24df48a49c9432288dcb089e P 9b6fe2f6be6a45f699c3534e102d4dcf: Deleting consensus metadata
I20260810 14:53:19.048182 378 catalog_manager.cc:4841] TS 493b4c03e00a4783aef988e4b268237f (127.30.74.67:38345): tablet 5a07ea2a24df48a49c9432288dcb089e (table test-table [id=8a5c17212b9743d78f71d0fa1847e966]) successfully deleted
I20260810 14:53:19.048194 377 catalog_manager.cc:4841] TS 9b6fe2f6be6a45f699c3534e102d4dcf (127.30.74.66:34207): tablet 5a07ea2a24df48a49c9432288dcb089e (table test-table [id=8a5c17212b9743d78f71d0fa1847e966]) successfully deleted
W20260810 14:53:19.050876 376 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:38517 (error 108)
I20260810 14:53:19.051124 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 619
I20260810 14:53:19.052418 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:19.051287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:38517 (local address 127.0.0.1:42358)
0810 14:53:19.052241 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:19.052353 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:38517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":933,"thread_start_us":58,"threads_started":1}
W20260810 14:53:19.052522 376 catalog_manager.cc:4584] TS 8cf6044aea834d71be137a3e7936b43d (127.30.74.65:38517): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5a07ea2a24df48a49c9432288dcb089e: Network error: Client connection negotiation failed: client connection to 127.30.74.65:38517: connect: Connection refused (error 111)
I20260810 14:53:19.059374 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 769
I20260810 14:53:19.066123 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 359
2026-08-10T14:53:19Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5810 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:53:19 dist-test-slave-0b58 krb5kdc[1083](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:53:19 dist-test-slave-0b58 krb5kdc[1083](info): set up 2 sockets
Aug 10 14:53:19 dist-test-slave-0b58 krb5kdc[1083](info): commencing operation
krb5kdc: starting...
W20260810 14:53:20.146647 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:53:20 dist-test-slave-0b58 krb5kdc[1083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:53:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:53:20.274704 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:45543
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:45543
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:20.357470 1097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:20.357738 1097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:20.357775 1097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:20.358959 1097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:20.359032 1097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:20.359056 1097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:20.359079 1097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:20.359109 1097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:20.360674 1097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:45543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:45543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:20.360968 1097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:20.361207 1097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:20.366459 1103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:20.366447 1106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:20.366447 1102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:20.366475 1097 server_base.cc:1034] running on GCE node
I20260810 14:53:20.366871 1097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:20.367210 1097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:20.368350 1097 hybrid_clock.cc:648] HybridClock initialized: now 1786373600368321 us; error 36 us; skew 500 ppm
Aug 10 14:53:20 dist-test-slave-0b58 krb5kdc[1083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373600, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:20.376981 1097 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:53:20.377415 1097 webserver.cc:460] Webserver started at http://127.30.74.126:37209/ using document root <none> and password file <none>
I20260810 14:53:20.377641 1097 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:20.377683 1097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:20.377772 1097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:20.378813 1097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "90314b0fb0104a35928028f17abe576b"
format_stamp: "Formatted at 2026-08-10 14:53:20 on dist-test-slave-0b58"
server_key: "0d0991a0ffd5114ff93df5e7919d1f0c"
server_key_iv: "25ea3fe1db22ae19a34a0e029c3ea376"
server_key_version: "encryptionkey@0"
I20260810 14:53:20.379155 1097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "90314b0fb0104a35928028f17abe576b"
format_stamp: "Formatted at 2026-08-10 14:53:20 on dist-test-slave-0b58"
server_key: "0d0991a0ffd5114ff93df5e7919d1f0c"
server_key_iv: "25ea3fe1db22ae19a34a0e029c3ea376"
server_key_version: "encryptionkey@0"
I20260810 14:53:20.380714 1097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:53:20.381498 1114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:20.381733 1097 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:20.381835 1097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "90314b0fb0104a35928028f17abe576b"
format_stamp: "Formatted at 2026-08-10 14:53:20 on dist-test-slave-0b58"
server_key: "0d0991a0ffd5114ff93df5e7919d1f0c"
server_key_iv: "25ea3fe1db22ae19a34a0e029c3ea376"
server_key_version: "encryptionkey@0"
I20260810 14:53:20.381915 1097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:20.397747 1097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:20.399645 1097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:20.399804 1097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:20.404091 1097 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:45543
I20260810 14:53:20.404160 1170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:45543 every 8 connection(s)
I20260810 14:53:20.404553 1097 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:20.405560 1171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:20.407843 1171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b: Bootstrap starting.
I20260810 14:53:20.408488 1171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:20.408742 1171 log.cc:824] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:20.409543 1171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b: No bootstrap required, opened a new log
I20260810 14:53:20.410121 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 1097
I20260810 14:53:20.410231 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:20.410408 31017 external_mini_cluster.cc:1428] Setting key 2723bb8ad5ff3b65d317dfcdbbb73526
I20260810 14:53:20.411705 1171 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90314b0fb0104a35928028f17abe576b" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45543 } }
I20260810 14:53:20.411896 1171 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:20.411940 1171 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90314b0fb0104a35928028f17abe576b, State: Initialized, Role: FOLLOWER
I20260810 14:53:20.412091 1171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90314b0fb0104a35928028f17abe576b" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45543 } }
I20260810 14:53:20.412180 1171 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:20.412216 1171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:20.412307 1171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:20.413352 1171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90314b0fb0104a35928028f17abe576b" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45543 } }
I20260810 14:53:20.413480 1171 leader_election.cc:302] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90314b0fb0104a35928028f17abe576b; no voters:
I20260810 14:53:20.413671 1171 leader_election.cc:288] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:20.413717 1176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:20.413902 1176 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [term 1 LEADER]: Becoming Leader. State: Replica: 90314b0fb0104a35928028f17abe576b, State: Running, Role: LEADER
I20260810 14:53:20.413954 1171 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:20.414038 1176 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90314b0fb0104a35928028f17abe576b" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45543 } }
I20260810 14:53:20.414597 1177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90314b0fb0104a35928028f17abe576b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90314b0fb0104a35928028f17abe576b" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45543 } } }
I20260810 14:53:20.414695 1177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:20.414852 1178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90314b0fb0104a35928028f17abe576b. Latest consensus state: current_term: 1 leader_uuid: "90314b0fb0104a35928028f17abe576b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90314b0fb0104a35928028f17abe576b" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45543 } } }
I20260810 14:53:20.414901 1178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:20.415373 1185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:20.415906 1185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:20.418112 1185 catalog_manager.cc:1347] Generated new cluster ID: 0524c0647d014fcd8f47f66eceafe0be
I20260810 14:53:20.418170 1185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 14:53:20 dist-test-slave-0b58 krb5kdc[1083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:20.422539 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:20.411000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59374 (local address 127.30.74.126:45543)
0810 14:53:20.411214 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:20.411222 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:20.411249 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:53:20.411680 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:20.411687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:20.411734 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:20.411851 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:20.412725 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:20.413824 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:20.414987 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:20.415945 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:20.416520 (+ 575us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:20.418931 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:20.418936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:20.418936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:20.418949 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:20.420826 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:20.421276 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:20.421280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:20.421280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:20.421394 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:20.421619 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:20.421620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:20.421621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:20.421880 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:20.422077 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:20.422180 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 14:53:20.422331 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":117,"threads_started":1}
I20260810 14:53:20.456609 1185 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:20.457226 1185 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:20.463562 1185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90314b0fb0104a35928028f17abe576b: Generated new TSK 0
I20260810 14:53:20.463788 1185 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:53:20.502902 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:45543
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:20.616163 1197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:20.616453 1197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:20.616480 1197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:20.617648 1197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:20.617690 1197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:20.617717 1197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:53:20.619230 1197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:45543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:20.619496 1197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:20.619745 1197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:20.624831 1204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:20.624840 1202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:20.624845 1206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:20.625231 1197 server_base.cc:1034] running on GCE node
I20260810 14:53:20.625459 1197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:20.625660 1197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:20.626809 1197 hybrid_clock.cc:648] HybridClock initialized: now 1786373600626800 us; error 32 us; skew 500 ppm
Aug 10 14:53:20 dist-test-slave-0b58 krb5kdc[1083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373600, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:20.629325 1197 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:53:20.629825 1197 webserver.cc:460] Webserver started at http://127.30.74.65:45419/ using document root <none> and password file <none>
I20260810 14:53:20.630070 1197 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:20.630142 1197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:20.630292 1197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:20.631351 1197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "d788e1bf38f145bf8b3ec22777725081"
format_stamp: "Formatted at 2026-08-10 14:53:20 on dist-test-slave-0b58"
server_key: "83a749018b5a5d83b32e31e71e424191"
server_key_iv: "1ac925899093275d5ce630d1304ad89b"
server_key_version: "encryptionkey@0"
I20260810 14:53:20.631713 1197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "d788e1bf38f145bf8b3ec22777725081"
format_stamp: "Formatted at 2026-08-10 14:53:20 on dist-test-slave-0b58"
server_key: "83a749018b5a5d83b32e31e71e424191"
server_key_iv: "1ac925899093275d5ce630d1304ad89b"
server_key_version: "encryptionkey@0"
I20260810 14:53:20.633298 1197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:53:20.634296 1213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:20.634451 1197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:20.634545 1197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "d788e1bf38f145bf8b3ec22777725081"
format_stamp: "Formatted at 2026-08-10 14:53:20 on dist-test-slave-0b58"
server_key: "83a749018b5a5d83b32e31e71e424191"
server_key_iv: "1ac925899093275d5ce630d1304ad89b"
server_key_version: "encryptionkey@0"
I20260810 14:53:20.634603 1197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:20.654969 1197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:20.657392 1197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:20.657513 1197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:20.657713 1197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:20.658049 1197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:20.658094 1197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:20.658118 1197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:20.658174 1197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:20.663856 1197 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:36697
I20260810 14:53:20.663935 1335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:36697 every 8 connection(s)
I20260810 14:53:20.664300 1197 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
Aug 10 14:53:20 dist-test-slave-0b58 krb5kdc[1083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373600, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:20.670181 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 1197
I20260810 14:53:20.670289 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:53:20.670476 31017 external_mini_cluster.cc:1428] Setting key a98d632ba17077a999041bcd34686bbb
I20260810 14:53:20.673834 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:20.665088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:55171 (local address 127.30.74.126:45543)
0810 14:53:20.665193 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:20.665196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:20.665366 (+ 170us) server_negotiation.cc:407] Connection header received
0810 14:53:20.665819 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:20.665820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:20.665836 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:20.665896 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:20.666816 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:20.667320 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:20.668323 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:20.668542 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:20.668600 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:20.670822 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:20.670823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:20.670823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:20.670832 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:20.672404 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:20.672823 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:20.672824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:20.672824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:20.672884 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:20.673160 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:20.673161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:20.673162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:20.673285 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:20.673357 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:20.673649 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 14:53:20.673741 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 14:53:20.674031 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:20.665033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45543 (local address 127.30.74.65:55171)
0810 14:53:20.665225 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:20.665273 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:20.665734 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:20.665951 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:20.665954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:20.666529 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:20.666721 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:20.666725 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:20.667400 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:20.667400 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:20.668191 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:20.668193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:20.668616 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:20.668617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:20.668728 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:20.669173 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:20.669181 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:20.670682 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:20.672488 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:20.672495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:20.672498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:20.672738 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:20.672967 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:20.672967 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:20.672968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:20.673083 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:20.673360 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:20.673367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:20.673508 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 14:53:20.673620 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 14:53:20.673784 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":113,"threads_started":1}
I20260810 14:53:20.674592 1336 heartbeater.cc:347] Connected to a master server at 127.30.74.126:45543
I20260810 14:53:20.674712 1336 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:20.674906 1336 heartbeater.cc:511] Master 127.30.74.126:45543 requested a full tablet report, sending...
I20260810 14:53:20.675458 1132 ts_manager.cc:194] Registered new tserver with Master: d788e1bf38f145bf8b3ec22777725081 (127.30.74.65:36697)
I20260810 14:53:20.676162 1132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:55171
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:53:20.698786 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:45543
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:20.782079 1343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:20.782335 1343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:20.782379 1343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:20.783578 1343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:20.783648 1343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:20.783694 1343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:53:20.785454 1343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:45543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:20.785742 1343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:20.785980 1343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:20.790984 1349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:20.790992 1348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:20.790998 1351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:20.791277 1343 server_base.cc:1034] running on GCE node
I20260810 14:53:20.791607 1343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:20.791872 1343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:20.793015 1343 hybrid_clock.cc:648] HybridClock initialized: now 1786373600792982 us; error 52 us; skew 500 ppm
Aug 10 14:53:20 dist-test-slave-0b58 krb5kdc[1083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373600, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:20.795608 1343 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:53:20.796152 1343 webserver.cc:460] Webserver started at http://127.30.74.66:42959/ using document root <none> and password file <none>
I20260810 14:53:20.796454 1343 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:20.796526 1343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:20.796686 1343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:20.797839 1343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "66b04b28f3d34f2da1c313a645595f37"
format_stamp: "Formatted at 2026-08-10 14:53:20 on dist-test-slave-0b58"
server_key: "928a6fc69e7f6f08629a49e88ed75797"
server_key_iv: "4a44a7b1ac84c33985b5d4fb6e2b617f"
server_key_version: "encryptionkey@0"
I20260810 14:53:20.798213 1343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "66b04b28f3d34f2da1c313a645595f37"
format_stamp: "Formatted at 2026-08-10 14:53:20 on dist-test-slave-0b58"
server_key: "928a6fc69e7f6f08629a49e88ed75797"
server_key_iv: "4a44a7b1ac84c33985b5d4fb6e2b617f"
server_key_version: "encryptionkey@0"
I20260810 14:53:20.799582 1343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:53:20.800359 1358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:20.800537 1343 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:20.800652 1343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "66b04b28f3d34f2da1c313a645595f37"
format_stamp: "Formatted at 2026-08-10 14:53:20 on dist-test-slave-0b58"
server_key: "928a6fc69e7f6f08629a49e88ed75797"
server_key_iv: "4a44a7b1ac84c33985b5d4fb6e2b617f"
server_key_version: "encryptionkey@0"
I20260810 14:53:20.800732 1343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:20.824860 1343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:20.827410 1343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:20.827589 1343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:20.827853 1343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:20.828188 1343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:20.828241 1343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:20.828306 1343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:20.828354 1343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:20.834687 1343 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:44279
I20260810 14:53:20.834800 1483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:44279 every 8 connection(s)
I20260810 14:53:20.835103 1343 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
Aug 10 14:53:20 dist-test-slave-0b58 krb5kdc[1083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373600, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:20.844683 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:20.835910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:56307 (local address 127.30.74.126:45543)
0810 14:53:20.835961 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:20.835963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:20.836153 (+ 190us) server_negotiation.cc:407] Connection header received
0810 14:53:20.836727 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:20.836728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:20.836738 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:20.836804 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:20.837841 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:20.838343 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:20.839422 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:20.839620 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:20.839746 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:20.841951 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:20.841952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:20.841953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:20.841961 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:20.843163 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:20.843565 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:20.843566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:20.843567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:20.843632 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:20.843938 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:20.843939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:20.843939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:20.844063 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:20.844144 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:20.844498 (+ 354us) server_negotiation.cc:299] Negotiation successful
0810 14:53:20.844589 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:53:20.844885 1486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:20.835874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45543 (local address 127.30.74.66:56307)
0810 14:53:20.836033 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:20.836076 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:20.836628 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:20.836856 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:20.836860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:20.837540 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:20.837744 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:20.837749 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:20.838480 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:20.838481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:20.839317 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:20.839319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:20.839759 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:20.839760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:20.839901 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:20.840474 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:20.840482 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:20.841846 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:20.843250 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:20.843256 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:20.843259 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:20.843485 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:20.843756 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:20.843757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:20.843757 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:20.843865 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:20.844135 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:20.844142 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:20.844349 (+ 207us) client_negotiation.cc:769] Sending connection context
0810 14:53:20.844463 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 14:53:20.844635 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260810 14:53:20.844887 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 1343
I20260810 14:53:20.844993 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:53:20.845184 31017 external_mini_cluster.cc:1428] Setting key b8a045ecb455452248b063c2a4fd7dbd
I20260810 14:53:20.845651 1484 heartbeater.cc:347] Connected to a master server at 127.30.74.126:45543
I20260810 14:53:20.845767 1484 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:20.845960 1484 heartbeater.cc:511] Master 127.30.74.126:45543 requested a full tablet report, sending...
I20260810 14:53:20.846408 1132 ts_manager.cc:194] Registered new tserver with Master: 66b04b28f3d34f2da1c313a645595f37 (127.30.74.66:44279)
I20260810 14:53:20.846823 1132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:56307
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:53:20.871083 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:45543
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:20.956738 1489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:20.956952 1489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:20.956979 1489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:20.958102 1489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:20.958137 1489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:20.958163 1489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:53:20.959563 1489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:45543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:20.959762 1489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:20.959945 1489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:20.964993 1494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:20.965005 1495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:20.965200 1489 server_base.cc:1034] running on GCE node
W20260810 14:53:20.965083 1497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:20.965495 1489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:20.965730 1489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:20.966912 1489 hybrid_clock.cc:648] HybridClock initialized: now 1786373600966903 us; error 32 us; skew 500 ppm
Aug 10 14:53:20 dist-test-slave-0b58 krb5kdc[1083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373600, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:20.969442 1489 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:53:20.969906 1489 webserver.cc:460] Webserver started at http://127.30.74.67:33221/ using document root <none> and password file <none>
I20260810 14:53:20.970162 1489 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:20.970238 1489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:20.970395 1489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:20.971544 1489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "8fdc69f62cbc41e899bf352b8d889d6d"
format_stamp: "Formatted at 2026-08-10 14:53:20 on dist-test-slave-0b58"
server_key: "e23e601ee31cadb76698974d2235e840"
server_key_iv: "8d39b147b76770a87ab79b6a02ed9b5f"
server_key_version: "encryptionkey@0"
I20260810 14:53:20.971922 1489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "8fdc69f62cbc41e899bf352b8d889d6d"
format_stamp: "Formatted at 2026-08-10 14:53:20 on dist-test-slave-0b58"
server_key: "e23e601ee31cadb76698974d2235e840"
server_key_iv: "8d39b147b76770a87ab79b6a02ed9b5f"
server_key_version: "encryptionkey@0"
I20260810 14:53:20.973438 1489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:53:20.974217 1505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:20.974409 1489 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:20.974519 1489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "8fdc69f62cbc41e899bf352b8d889d6d"
format_stamp: "Formatted at 2026-08-10 14:53:20 on dist-test-slave-0b58"
server_key: "e23e601ee31cadb76698974d2235e840"
server_key_iv: "8d39b147b76770a87ab79b6a02ed9b5f"
server_key_version: "encryptionkey@0"
I20260810 14:53:20.974607 1489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:20.989706 1489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:20.991925 1489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:20.992093 1489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:20.992396 1489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:20.992738 1489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:20.992801 1489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:20.992846 1489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:20.992872 1489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:20.998986 1489 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:33633
I20260810 14:53:20.999038 1627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:33633 every 8 connection(s)
I20260810 14:53:20.999396 1489 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
Aug 10 14:53:21 dist-test-slave-0b58 krb5kdc[1083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373600, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:21.006880 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 1489
I20260810 14:53:21.007010 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:53:21.007262 31017 external_mini_cluster.cc:1428] Setting key c8144a34c936879d4cb2bd67081fc26a
I20260810 14:53:21.009644 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:21.000304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:48597 (local address 127.30.74.126:45543)
0810 14:53:21.000362 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:21.000366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:21.000553 (+ 187us) server_negotiation.cc:407] Connection header received
0810 14:53:21.001063 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:21.001064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:21.001109 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:21.001172 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:21.002137 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:21.002627 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:21.003709 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:21.003902 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:21.003959 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:21.006227 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:21.006229 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:21.006229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:21.006236 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:21.008040 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:21.008583 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:21.008585 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:21.008586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:21.008654 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:21.008929 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:21.008930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:21.008930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:21.009056 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:21.009128 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:21.009441 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 14:53:21.009551 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 14:53:21.009837 1630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:21.000174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45543 (local address 127.30.74.67:48597)
0810 14:53:21.000416 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:21.000459 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:21.000945 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:21.001251 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:21.001254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:21.001835 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:21.002047 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:21.002051 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:21.002717 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:21.002718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:21.003628 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:21.003631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:21.003979 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:21.003980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:21.004090 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:21.004655 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:21.004666 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:21.006131 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:21.008124 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:21.008134 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:21.008136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:21.008497 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:21.008725 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:21.008726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:21.008726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:21.008854 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:21.009132 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:21.009140 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:21.009297 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 14:53:21.009402 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 14:53:21.009583 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":118,"threads_started":1}
I20260810 14:53:21.010500 1628 heartbeater.cc:347] Connected to a master server at 127.30.74.126:45543
I20260810 14:53:21.010622 1628 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:21.010880 1628 heartbeater.cc:511] Master 127.30.74.126:45543 requested a full tablet report, sending...
I20260810 14:53:21.011374 1132 ts_manager.cc:194] Registered new tserver with Master: 8fdc69f62cbc41e899bf352b8d889d6d (127.30.74.67:33633)
I20260810 14:53:21.011788 1132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:48597
I20260810 14:53:21.019562 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 14:53:21.032058 1175 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:21.030967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34514 (local address 127.30.74.126:45543)
0810 14:53:21.031066 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:21.031069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:21.031178 (+ 109us) server_negotiation.cc:407] Connection header received
0810 14:53:21.031292 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:21.031293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:21.031303 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:21.031367 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:21.031978 (+ 611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34514: BlockingRecv error: recv got EOF from 127.0.0.1:34514 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:53:21.033053 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 1197
I20260810 14:53:21.039198 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 1343
I20260810 14:53:21.047497 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 1489
I20260810 14:53:21.055737 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 1097
2026-08-10T14:53:21Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1987 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:53:21 dist-test-slave-0b58 krb5kdc[1640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:53:21 dist-test-slave-0b58 krb5kdc[1640](info): set up 2 sockets
Aug 10 14:53:21 dist-test-slave-0b58 krb5kdc[1640](info): commencing operation
krb5kdc: starting...
W20260810 14:53:22.138904 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:53:22 dist-test-slave-0b58 krb5kdc[1640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:53:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:53:22.265420 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:35253
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:44605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:35253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:22.349061 1656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:22.349301 1656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:22.349328 1656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:22.350521 1656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:22.350564 1656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:22.350577 1656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:22.350591 1656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:22.350603 1656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:22.352131 1656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:44605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:35253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:35253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:22.352432 1656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:22.352644 1656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:22.357635 1664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:22.357638 1661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:22.357668 1662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:22.358466 1656 server_base.cc:1034] running on GCE node
I20260810 14:53:22.358752 1656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:22.359086 1656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:22.360335 1656 hybrid_clock.cc:648] HybridClock initialized: now 1786373602360322 us; error 41 us; skew 500 ppm
Aug 10 14:53:22 dist-test-slave-0b58 krb5kdc[1640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373602, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:22.362978 1656 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:53:22.363456 1656 webserver.cc:460] Webserver started at http://127.30.74.126:43001/ using document root <none> and password file <none>
I20260810 14:53:22.363713 1656 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:22.363793 1656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:22.363946 1656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:22.365136 1656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "ccbe2c0303c1481eb6f312ba87f45854"
format_stamp: "Formatted at 2026-08-10 14:53:22 on dist-test-slave-0b58"
server_key: "257c02f36070897c779fd8d2b2a9c67d"
server_key_iv: "5b501fceecc9f102201b9308475ee4b2"
server_key_version: "encryptionkey@0"
I20260810 14:53:22.365527 1656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "ccbe2c0303c1481eb6f312ba87f45854"
format_stamp: "Formatted at 2026-08-10 14:53:22 on dist-test-slave-0b58"
server_key: "257c02f36070897c779fd8d2b2a9c67d"
server_key_iv: "5b501fceecc9f102201b9308475ee4b2"
server_key_version: "encryptionkey@0"
I20260810 14:53:22.367012 1656 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:53:22.367841 1671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:22.368063 1656 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:22.368182 1656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "ccbe2c0303c1481eb6f312ba87f45854"
format_stamp: "Formatted at 2026-08-10 14:53:22 on dist-test-slave-0b58"
server_key: "257c02f36070897c779fd8d2b2a9c67d"
server_key_iv: "5b501fceecc9f102201b9308475ee4b2"
server_key_version: "encryptionkey@0"
I20260810 14:53:22.368289 1656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:22.390161 1656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:22.392635 1656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:22.392827 1656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:22.396934 1656 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:35253
I20260810 14:53:22.396972 1724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:35253 every 8 connection(s)
I20260810 14:53:22.397487 1656 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:22.398243 1725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:22.400606 1725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854: Bootstrap starting.
I20260810 14:53:22.401108 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 1656
I20260810 14:53:22.401238 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:22.401294 1725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:22.401466 31017 external_mini_cluster.cc:1428] Setting key 0f5628d94a5aa3565db5f2f89883ec57
I20260810 14:53:22.401652 1725 log.cc:824] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:22.402532 1725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854: No bootstrap required, opened a new log
I20260810 14:53:22.404978 1725 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbe2c0303c1481eb6f312ba87f45854" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 35253 } }
I20260810 14:53:22.405172 1725 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:22.405220 1725 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbe2c0303c1481eb6f312ba87f45854, State: Initialized, Role: FOLLOWER
I20260810 14:53:22.405357 1725 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbe2c0303c1481eb6f312ba87f45854" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 35253 } }
I20260810 14:53:22.405435 1725 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:22.405465 1725 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:22.405503 1725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:22.406229 1725 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbe2c0303c1481eb6f312ba87f45854" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 35253 } }
I20260810 14:53:22.406325 1725 leader_election.cc:302] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccbe2c0303c1481eb6f312ba87f45854; no voters:
I20260810 14:53:22.406543 1725 leader_election.cc:288] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:22.406570 1731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:22.406705 1731 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [term 1 LEADER]: Becoming Leader. State: Replica: ccbe2c0303c1481eb6f312ba87f45854, State: Running, Role: LEADER
I20260810 14:53:22.406839 1725 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:22.406847 1731 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbe2c0303c1481eb6f312ba87f45854" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 35253 } }
I20260810 14:53:22.407466 1733 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccbe2c0303c1481eb6f312ba87f45854. Latest consensus state: current_term: 1 leader_uuid: "ccbe2c0303c1481eb6f312ba87f45854" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbe2c0303c1481eb6f312ba87f45854" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 35253 } } }
I20260810 14:53:22.407580 1733 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:22.407768 1732 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccbe2c0303c1481eb6f312ba87f45854" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbe2c0303c1481eb6f312ba87f45854" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 35253 } } }
I20260810 14:53:22.407893 1732 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:22.407858 1741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:22.408667 1741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 14:53:22 dist-test-slave-0b58 krb5kdc[1640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:22.410755 1741 catalog_manager.cc:1347] Generated new cluster ID: dc04d603cafd4a38a1aa7662bf6dd534
I20260810 14:53:22.410809 1741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:53:22.414525 1730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:22.401996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35952 (local address 127.30.74.126:35253)
0810 14:53:22.402183 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:22.402194 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:22.402233 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:53:22.402767 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:22.402775 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:22.402826 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:22.402951 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:22.403849 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:22.404871 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:22.405972 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:22.406571 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:22.407141 (+ 570us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:22.410236 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:22.410240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:22.410241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:22.410258 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:22.412555 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:22.413034 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:22.413037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:22.413038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:22.413155 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:22.413389 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:22.413390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:22.413390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:22.413735 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:22.413899 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:22.414020 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 14:53:22.414300 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260810 14:53:22.415565 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:53:22.428210 1741 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:22.428856 1741 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:22.439489 1741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccbe2c0303c1481eb6f312ba87f45854: Generated new TSK 0
I20260810 14:53:22.439765 1741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:53:22.444205 1730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:22.440028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35960 (local address 127.30.74.126:35253)
0810 14:53:22.440079 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:22.440082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:22.440455 (+ 373us) server_negotiation.cc:407] Connection header received
0810 14:53:22.440538 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:22.440539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:22.440552 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:22.440610 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:22.441052 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:22.441651 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:22.442508 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:22.442761 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:22.442847 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:22.443674 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:22.443676 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:22.443676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:53:22.443893 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:22.443912 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:53:22.443918 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:22.443996 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:22.444061 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 14:53:22.444092 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 14:53:22.447490 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 1656
2026-08-10T14:53:22Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1389 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:53:22 dist-test-slave-0b58 krb5kdc[1762](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:53:22 dist-test-slave-0b58 krb5kdc[1762](info): set up 2 sockets
Aug 10 14:53:22 dist-test-slave-0b58 krb5kdc[1762](info): commencing operation
krb5kdc: starting...
W20260810 14:53:23.571096 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:53:23 dist-test-slave-0b58 krb5kdc[1762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:53:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:53:23.699163 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:43601
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:43965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:43601
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:23.783030 1776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:23.783268 1776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:23.783293 1776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:23.784608 1776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:23.784657 1776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:23.784672 1776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:23.784684 1776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:23.784696 1776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:23.786299 1776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:43965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:43601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:43601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:23.786566 1776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:23.786803 1776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:23.791853 1782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:23.791944 1776 server_base.cc:1034] running on GCE node
W20260810 14:53:23.791958 1781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:23.791853 1785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:23.792338 1776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:23.792636 1776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:23.793800 1776 hybrid_clock.cc:648] HybridClock initialized: now 1786373603793794 us; error 38 us; skew 500 ppm
Aug 10 14:53:23 dist-test-slave-0b58 krb5kdc[1762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373603, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:23.796443 1776 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:53:23.796893 1776 webserver.cc:460] Webserver started at http://127.30.74.126:38433/ using document root <none> and password file <none>
I20260810 14:53:23.797132 1776 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:23.797210 1776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:23.797355 1776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:23.798465 1776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "91bdd57502b6433983a49cd17b3babda"
format_stamp: "Formatted at 2026-08-10 14:53:23 on dist-test-slave-0b58"
server_key: "9807d3b705ecb40bbc84e00ad504d805"
server_key_iv: "a8d0d2b66ccab16b0fc2689d80899079"
server_key_version: "encryptionkey@0"
I20260810 14:53:23.798856 1776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "91bdd57502b6433983a49cd17b3babda"
format_stamp: "Formatted at 2026-08-10 14:53:23 on dist-test-slave-0b58"
server_key: "9807d3b705ecb40bbc84e00ad504d805"
server_key_iv: "a8d0d2b66ccab16b0fc2689d80899079"
server_key_version: "encryptionkey@0"
I20260810 14:53:23.800307 1776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 14:53:23.801162 1793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:23.801379 1776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:23.801488 1776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "91bdd57502b6433983a49cd17b3babda"
format_stamp: "Formatted at 2026-08-10 14:53:23 on dist-test-slave-0b58"
server_key: "9807d3b705ecb40bbc84e00ad504d805"
server_key_iv: "a8d0d2b66ccab16b0fc2689d80899079"
server_key_version: "encryptionkey@0"
I20260810 14:53:23.801576 1776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:23.809048 1776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:23.811214 1776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:23.811398 1776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:23.815594 1776 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:43601
I20260810 14:53:23.815627 1848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:43601 every 8 connection(s)
I20260810 14:53:23.816000 1776 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:23.817075 1849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:23.819382 1849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda: Bootstrap starting.
I20260810 14:53:23.820011 1849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:23.820349 1849 log.cc:824] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:23.821110 1849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda: No bootstrap required, opened a new log
I20260810 14:53:23.822713 1849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91bdd57502b6433983a49cd17b3babda" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43601 } }
I20260810 14:53:23.822857 1849 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:23.822881 1849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91bdd57502b6433983a49cd17b3babda, State: Initialized, Role: FOLLOWER
I20260810 14:53:23.823047 1849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91bdd57502b6433983a49cd17b3babda" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43601 } }
I20260810 14:53:23.823133 1849 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:23.823158 1849 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:23.823190 1849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:23.824057 1849 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91bdd57502b6433983a49cd17b3babda" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43601 } }
I20260810 14:53:23.824148 1849 leader_election.cc:302] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91bdd57502b6433983a49cd17b3babda; no voters:
I20260810 14:53:23.824411 1849 leader_election.cc:288] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:23.824491 1853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:23.824651 1853 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [term 1 LEADER]: Becoming Leader. State: Replica: 91bdd57502b6433983a49cd17b3babda, State: Running, Role: LEADER
I20260810 14:53:23.824700 1849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:23.824749 1853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91bdd57502b6433983a49cd17b3babda" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43601 } }
I20260810 14:53:23.824918 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 1776
I20260810 14:53:23.825023 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:23.825227 31017 external_mini_cluster.cc:1428] Setting key b22df99d2fc69e2196aeca20ff2ef22f
I20260810 14:53:23.825235 1854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91bdd57502b6433983a49cd17b3babda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91bdd57502b6433983a49cd17b3babda" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43601 } } }
I20260810 14:53:23.825317 1854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:23.825829 1855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91bdd57502b6433983a49cd17b3babda. Latest consensus state: current_term: 1 leader_uuid: "91bdd57502b6433983a49cd17b3babda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91bdd57502b6433983a49cd17b3babda" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43601 } } }
I20260810 14:53:23.825938 1855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:23.826083 1858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:23.826656 1858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:23.828337 1858 catalog_manager.cc:1347] Generated new cluster ID: 99e5cc5567e340d39a4e73f27ffbb743
I20260810 14:53:23.828392 1858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 14:53:23 dist-test-slave-0b58 krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:23.836387 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:23.825890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33442 (local address 127.30.74.126:43601)
0810 14:53:23.826129 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:23.826140 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:23.826364 (+ 224us) server_negotiation.cc:407] Connection header received
0810 14:53:23.826607 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:23.826615 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:23.826676 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:23.826823 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:23.827775 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:23.828953 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:23.829745 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:23.830199 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:23.830320 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:23.832776 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:23.832781 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:23.832781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:23.832795 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:23.834630 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:23.835030 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:23.835034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:23.835035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:23.835174 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:23.835404 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:23.835405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:23.835405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:23.835677 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:23.835847 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:23.835969 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 14:53:23.836125 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":75,"threads_started":1}
I20260810 14:53:23.846861 1858 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:23.847661 1858 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:23.850425 1858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91bdd57502b6433983a49cd17b3babda: Generated new TSK 0
I20260810 14:53:23.850612 1858 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:53:23.883414 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43601
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:43965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:23.967649 1876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:23.967891 1876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:23.967931 1876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:23.969240 1876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:23.969309 1876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:23.969353 1876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:53:23.970906 1876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:43965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:23.971200 1876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:23.971450 1876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:23.976579 1882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:23.976672 1884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:23.976789 1881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:23.976990 1876 server_base.cc:1034] running on GCE node
I20260810 14:53:23.977285 1876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:23.977568 1876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:23.978773 1876 hybrid_clock.cc:648] HybridClock initialized: now 1786373603978772 us; error 48 us; skew 500 ppm
Aug 10 14:53:23 dist-test-slave-0b58 krb5kdc[1762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373603, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:23.981395 1876 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:53:23.981873 1876 webserver.cc:460] Webserver started at http://127.30.74.65:36801/ using document root <none> and password file <none>
I20260810 14:53:23.982121 1876 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:23.982193 1876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:23.982359 1876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:23.983453 1876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "4f74cc1e1df249eeb9b001fe995747df"
format_stamp: "Formatted at 2026-08-10 14:53:23 on dist-test-slave-0b58"
server_key: "e5b425267a95f53d69344ebc22dcf85f"
server_key_iv: "27c3093852c9cbf845509abc9cb6c694"
server_key_version: "encryptionkey@0"
I20260810 14:53:23.983829 1876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "4f74cc1e1df249eeb9b001fe995747df"
format_stamp: "Formatted at 2026-08-10 14:53:23 on dist-test-slave-0b58"
server_key: "e5b425267a95f53d69344ebc22dcf85f"
server_key_iv: "27c3093852c9cbf845509abc9cb6c694"
server_key_version: "encryptionkey@0"
I20260810 14:53:23.985383 1876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:53:23.986203 1892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:23.986414 1876 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:23.986528 1876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "4f74cc1e1df249eeb9b001fe995747df"
format_stamp: "Formatted at 2026-08-10 14:53:23 on dist-test-slave-0b58"
server_key: "e5b425267a95f53d69344ebc22dcf85f"
server_key_iv: "27c3093852c9cbf845509abc9cb6c694"
server_key_version: "encryptionkey@0"
I20260810 14:53:23.986608 1876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:23.999758 1876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:24.002374 1876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:24.002578 1876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:24.002849 1876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:24.003203 1876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:24.003274 1876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:24.003319 1876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:24.003345 1876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:24.009537 1876 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:40747
I20260810 14:53:24.009621 2016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:40747 every 8 connection(s)
I20260810 14:53:24.009974 1876 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
Aug 10 14:53:24 dist-test-slave-0b58 krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373603, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:24.019845 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 1876
I20260810 14:53:24.019981 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:53:24.019989 2019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:24.010717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43601 (local address 127.30.74.65:51233)
0810 14:53:24.010883 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:24.010923 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:24.011464 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:24.011691 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:24.011695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:24.012321 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:24.012526 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:24.012531 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:24.013261 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:24.013262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:24.014093 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:24.014096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:24.014572 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:24.014573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:24.014700 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:24.015303 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:24.015312 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:24.016774 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:24.018554 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:24.018562 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:24.018564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:24.018802 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:24.019037 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:24.019038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:24.019038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:24.019142 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:24.019425 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:24.019430 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:24.019556 (+ 126us) client_negotiation.cc:769] Sending connection context
0810 14:53:24.019640 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 14:53:24.019771 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260810 14:53:24.020018 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:24.010807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:51233 (local address 127.30.74.126:43601)
0810 14:53:24.010897 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:24.010900 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:24.011009 (+ 109us) server_negotiation.cc:407] Connection header received
0810 14:53:24.011560 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:24.011561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:24.011574 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:24.011629 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:24.012614 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:24.013153 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:24.014245 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:24.014460 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:24.014545 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:24.016865 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:24.016867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:24.016867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:24.016874 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:24.018449 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:24.018903 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:24.018905 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:24.018905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:24.018968 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:24.019238 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:24.019239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:24.019239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:24.019364 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:24.019422 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:24.019795 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 14:53:24.019904 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 14:53:24.020183 31017 external_mini_cluster.cc:1428] Setting key cf9e0f0c50bfdf17431e649608f6d275
I20260810 14:53:24.020910 2017 heartbeater.cc:347] Connected to a master server at 127.30.74.126:43601
I20260810 14:53:24.021042 2017 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:24.021286 2017 heartbeater.cc:511] Master 127.30.74.126:43601 requested a full tablet report, sending...
I20260810 14:53:24.021850 1810 ts_manager.cc:194] Registered new tserver with Master: 4f74cc1e1df249eeb9b001fe995747df (127.30.74.65:40747)
I20260810 14:53:24.022526 1810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:51233
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:53:24.048072 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43601
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:43965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:24.132277 2022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:24.132547 2022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:24.132591 2022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:24.133795 2022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:24.133867 2022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:24.133913 2022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:53:24.135457 2022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:43965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:24.135761 2022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:24.136018 2022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:24.141314 2028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:24.141319 2030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:24.141319 2027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:24.141474 2022 server_base.cc:1034] running on GCE node
I20260810 14:53:24.141817 2022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:24.142099 2022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:24.143278 2022 hybrid_clock.cc:648] HybridClock initialized: now 1786373604143266 us; error 33 us; skew 500 ppm
Aug 10 14:53:24 dist-test-slave-0b58 krb5kdc[1762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373604, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:24.145885 2022 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:53:24.146397 2022 webserver.cc:460] Webserver started at http://127.30.74.66:44971/ using document root <none> and password file <none>
I20260810 14:53:24.146656 2022 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:24.146726 2022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:24.146885 2022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:24.147987 2022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "edee62ed925b40deb6afa124e2666edd"
format_stamp: "Formatted at 2026-08-10 14:53:24 on dist-test-slave-0b58"
server_key: "e52b86a283ab60e406eeab8850a328e0"
server_key_iv: "cfa847d7aeefe13d372ca046dc262237"
server_key_version: "encryptionkey@0"
I20260810 14:53:24.148432 2022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "edee62ed925b40deb6afa124e2666edd"
format_stamp: "Formatted at 2026-08-10 14:53:24 on dist-test-slave-0b58"
server_key: "e52b86a283ab60e406eeab8850a328e0"
server_key_iv: "cfa847d7aeefe13d372ca046dc262237"
server_key_version: "encryptionkey@0"
I20260810 14:53:24.149943 2022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:24.150903 2037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:24.151124 2022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:24.151243 2022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "edee62ed925b40deb6afa124e2666edd"
format_stamp: "Formatted at 2026-08-10 14:53:24 on dist-test-slave-0b58"
server_key: "e52b86a283ab60e406eeab8850a328e0"
server_key_iv: "cfa847d7aeefe13d372ca046dc262237"
server_key_version: "encryptionkey@0"
I20260810 14:53:24.151335 2022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:24.173220 2022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:24.175853 2022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:24.176043 2022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:24.176378 2022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:24.176739 2022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:24.176797 2022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:24.176842 2022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:24.176868 2022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:24.183145 2022 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:39937
I20260810 14:53:24.183194 2158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:39937 every 8 connection(s)
I20260810 14:53:24.183539 2022 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:53:24.184152 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 2022
I20260810 14:53:24.184295 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:53:24.184542 31017 external_mini_cluster.cc:1428] Setting key cf01ac88a9814ace2cc481a27a8902ca
Aug 10 14:53:24 dist-test-slave-0b58 krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373604, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
I20260810 14:53:24.194173 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:24.184379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:47575 (local address 127.30.74.126:43601)
0810 14:53:24.184465 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:24.184468 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:24.184755 (+ 287us) server_negotiation.cc:407] Connection header received
0810 14:53:24.185515 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:24.185516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:24.185529 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:24.185569 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:24.186599 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:24.187113 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:24.188173 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:24.188554 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:24.188639 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:24.191098 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:24.191099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:24.191100 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:24.191108 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:24.192674 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:24.193115 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:24.193117 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:24.193117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:24.193186 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:24.193497 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:24.193499 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:24.193499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:24.193672 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:24.193758 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:24.194021 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 14:53:24.194103 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 14:53:24.194361 2162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:24.184362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43601 (local address 127.30.74.66:47575)
0810 14:53:24.184543 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:24.184656 (+ 113us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:24.185402 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:24.185652 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:24.185657 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:24.186282 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:24.186491 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:24.186496 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:24.187222 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:24.187223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:24.188073 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:24.188076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:24.188648 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:24.188649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:24.188804 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:24.189413 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:24.189422 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:24.190980 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:24.192765 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:24.192774 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:24.192786 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:24.193032 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:24.193265 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:24.193266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:24.193266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:24.193385 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:24.193753 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:24.193759 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:24.193896 (+ 137us) client_negotiation.cc:769] Sending connection context
0810 14:53:24.193986 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 14:53:24.194119 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260810 14:53:24.194964 2159 heartbeater.cc:347] Connected to a master server at 127.30.74.126:43601
I20260810 14:53:24.195065 2159 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:24.195222 2159 heartbeater.cc:511] Master 127.30.74.126:43601 requested a full tablet report, sending...
I20260810 14:53:24.195664 1810 ts_manager.cc:194] Registered new tserver with Master: edee62ed925b40deb6afa124e2666edd (127.30.74.66:39937)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
I20260810 14:53:24.196069 1810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:47575
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:53:24.212513 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43601
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:43965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:24.300194 2169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:24.300455 2169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:24.300482 2169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:24.301651 2169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:24.301685 2169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:24.301712 2169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:53:24.303145 2169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:43965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:24.303354 2169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:24.303546 2169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:24.308831 2177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:24.308831 2174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:24.309072 2169 server_base.cc:1034] running on GCE node
W20260810 14:53:24.309266 2175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:24.309538 2169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:24.309832 2169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:24.311020 2169 hybrid_clock.cc:648] HybridClock initialized: now 1786373604311015 us; error 40 us; skew 500 ppm
Aug 10 14:53:24 dist-test-slave-0b58 krb5kdc[1762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373604, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:24.313874 2169 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:53:24.314402 2169 webserver.cc:460] Webserver started at http://127.30.74.67:43853/ using document root <none> and password file <none>
I20260810 14:53:24.314642 2169 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:24.314711 2169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:24.314875 2169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:24.315965 2169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "3c68eb492f6f4c81993f87ec98a5e059"
format_stamp: "Formatted at 2026-08-10 14:53:24 on dist-test-slave-0b58"
server_key: "89899481d3e3fbbadf86c88a03525e18"
server_key_iv: "8811c46a4b57aec9f3795f81d2d9f2a1"
server_key_version: "encryptionkey@0"
I20260810 14:53:24.316455 2169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "3c68eb492f6f4c81993f87ec98a5e059"
format_stamp: "Formatted at 2026-08-10 14:53:24 on dist-test-slave-0b58"
server_key: "89899481d3e3fbbadf86c88a03525e18"
server_key_iv: "8811c46a4b57aec9f3795f81d2d9f2a1"
server_key_version: "encryptionkey@0"
I20260810 14:53:24.317873 2169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:24.318703 2186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:24.318897 2169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:24.319002 2169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "3c68eb492f6f4c81993f87ec98a5e059"
format_stamp: "Formatted at 2026-08-10 14:53:24 on dist-test-slave-0b58"
server_key: "89899481d3e3fbbadf86c88a03525e18"
server_key_iv: "8811c46a4b57aec9f3795f81d2d9f2a1"
server_key_version: "encryptionkey@0"
I20260810 14:53:24.319090 2169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:24.330514 2169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:24.333120 2169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:24.333309 2169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:24.333575 2169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:24.333932 2169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:24.333988 2169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:24.334026 2169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:24.334074 2169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:24.340623 2169 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:43417
I20260810 14:53:24.340677 2311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:43417 every 8 connection(s)
I20260810 14:53:24.341015 2169 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
Aug 10 14:53:24 dist-test-slave-0b58 krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373604, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:24.348660 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 2169
I20260810 14:53:24.348774 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:53:24.348980 31017 external_mini_cluster.cc:1428] Setting key a3a3beabf9c9d190f5ace2a029787432
I20260810 14:53:24.351914 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:24.341880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:53575 (local address 127.30.74.126:43601)
0810 14:53:24.341978 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:24.341980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:24.342161 (+ 181us) server_negotiation.cc:407] Connection header received
0810 14:53:24.342724 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:24.342725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:24.342738 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:24.342812 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:24.343858 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:24.344477 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:24.345538 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:24.345754 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:24.345851 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:24.348313 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:24.348315 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:24.348316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:24.348323 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:24.350257 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:24.350850 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:24.350852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:24.350852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:24.350928 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:24.351204 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:24.351205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:24.351206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:24.351344 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:24.351423 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:24.351721 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 14:53:24.351813 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 14:53:24.352130 2314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:24.341791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43601 (local address 127.30.74.67:53575)
0810 14:53:24.342014 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:24.342068 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:24.342610 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:24.342889 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:24.342893 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:24.343512 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:24.343727 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:24.343732 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:24.344581 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:24.344583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:24.345432 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:24.345434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:24.345868 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:24.345869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:24.346009 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:24.346607 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:24.346616 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:24.348161 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:24.350381 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:24.350391 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:24.350393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:24.350731 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:24.351002 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:24.351003 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:24.351003 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:24.351130 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:24.351416 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:24.351424 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:24.351582 (+ 158us) client_negotiation.cc:769] Sending connection context
0810 14:53:24.351698 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 14:53:24.351888 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":96,"threads_started":1}
I20260810 14:53:24.352702 2312 heartbeater.cc:347] Connected to a master server at 127.30.74.126:43601
I20260810 14:53:24.352823 2312 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:24.353030 2312 heartbeater.cc:511] Master 127.30.74.126:43601 requested a full tablet report, sending...
I20260810 14:53:24.353451 1810 ts_manager.cc:194] Registered new tserver with Master: 3c68eb492f6f4c81993f87ec98a5e059 (127.30.74.67:43417)
I20260810 14:53:24.353886 1810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:53575
I20260810 14:53:24.361222 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:53:24 dist-test-slave-0b58 krb5kdc[1762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373604, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 14:53:24 dist-test-slave-0b58 krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373604, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:24.386435 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:24.378287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33458 (local address 127.30.74.126:43601)
0810 14:53:24.378378 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:24.378381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:24.378472 (+ 91us) server_negotiation.cc:407] Connection header received
0810 14:53:24.378584 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:24.378585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:24.378598 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:24.378669 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:24.379283 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:24.379792 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:24.380653 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:24.380861 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:24.380950 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:24.383284 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:24.383285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:24.383285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:24.383293 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:24.384781 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:24.385220 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:24.385221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:24.385221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:24.385336 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:24.385690 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:24.385691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:24.385692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:24.385978 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:24.386132 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:24.386288 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 14:53:24.386364 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 14:53:24.386893 1810 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33458
I20260810 14:53:24.387544 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 1876
I20260810 14:53:24.403103 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 2022
I20260810 14:53:24.412871 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 2169
I20260810 14:53:24.419603 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 1776
2026-08-10T14:53:24Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1973 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:53:24 dist-test-slave-0b58 krb5kdc[2324](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:53:24 dist-test-slave-0b58 krb5kdc[2324](info): set up 2 sockets
Aug 10 14:53:24 dist-test-slave-0b58 krb5kdc[2324](info): commencing operation
krb5kdc: starting...
W20260810 14:53:25.510131 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:53:25 dist-test-slave-0b58 krb5kdc[2324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:53:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:53:25.635282 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:44535
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:44535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:25.718484 2337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:25.718720 2337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:25.718744 2337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:25.719856 2337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:25.719892 2337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:25.719905 2337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:25.719918 2337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:25.719929 2337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:25.721511 2337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:44535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:44535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:25.721737 2337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:25.721920 2337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:25.727329 2342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:25.727321 2346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:25.727321 2343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:25.728299 2337 server_base.cc:1034] running on GCE node
I20260810 14:53:25.728616 2337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:25.728972 2337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:25.730150 2337 hybrid_clock.cc:648] HybridClock initialized: now 1786373605730132 us; error 36 us; skew 500 ppm
Aug 10 14:53:25 dist-test-slave-0b58 krb5kdc[2324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373605, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:25.737393 2337 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:53:25.737839 2337 webserver.cc:460] Webserver started at http://127.30.74.126:40007/ using document root <none> and password file <none>
I20260810 14:53:25.738090 2337 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:25.738137 2337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:25.738291 2337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:25.739401 2337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "38771d48f92a49e5bd7716d58d6ef662"
format_stamp: "Formatted at 2026-08-10 14:53:25 on dist-test-slave-0b58"
server_key: "a1029f45d53c48051ec6a66496708b0f"
server_key_iv: "3ff796748dc24faee207e1290861d0b6"
server_key_version: "encryptionkey@0"
I20260810 14:53:25.739776 2337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "38771d48f92a49e5bd7716d58d6ef662"
format_stamp: "Formatted at 2026-08-10 14:53:25 on dist-test-slave-0b58"
server_key: "a1029f45d53c48051ec6a66496708b0f"
server_key_iv: "3ff796748dc24faee207e1290861d0b6"
server_key_version: "encryptionkey@0"
I20260810 14:53:25.741290 2337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:53:25.742094 2353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:25.742352 2337 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:25.742461 2337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "38771d48f92a49e5bd7716d58d6ef662"
format_stamp: "Formatted at 2026-08-10 14:53:25 on dist-test-slave-0b58"
server_key: "a1029f45d53c48051ec6a66496708b0f"
server_key_iv: "3ff796748dc24faee207e1290861d0b6"
server_key_version: "encryptionkey@0"
I20260810 14:53:25.742545 2337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:25.767418 2337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:25.769587 2337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:25.769773 2337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:25.774008 2337 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:44535
I20260810 14:53:25.774049 2411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:44535 every 8 connection(s)
I20260810 14:53:25.774593 2337 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:25.775310 2413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:25.777541 2413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662: Bootstrap starting.
I20260810 14:53:25.778087 2413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:25.778324 2413 log.cc:824] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:25.778987 2413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662: No bootstrap required, opened a new log
I20260810 14:53:25.780558 2413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38771d48f92a49e5bd7716d58d6ef662" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44535 } }
I20260810 14:53:25.780705 2413 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:25.780730 2413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38771d48f92a49e5bd7716d58d6ef662, State: Initialized, Role: FOLLOWER
I20260810 14:53:25.780869 2413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38771d48f92a49e5bd7716d58d6ef662" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44535 } }
I20260810 14:53:25.780969 2413 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:25.781013 2413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:25.781066 2413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:25.781162 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 2337
I20260810 14:53:25.781282 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:25.781477 31017 external_mini_cluster.cc:1428] Setting key 8b28b56fff16622f34ec8c4ebc5aa125
I20260810 14:53:25.782296 2413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38771d48f92a49e5bd7716d58d6ef662" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44535 } }
I20260810 14:53:25.782387 2413 leader_election.cc:302] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38771d48f92a49e5bd7716d58d6ef662; no voters:
I20260810 14:53:25.782599 2413 leader_election.cc:288] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:25.782631 2418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:25.782745 2418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [term 1 LEADER]: Becoming Leader. State: Replica: 38771d48f92a49e5bd7716d58d6ef662, State: Running, Role: LEADER
I20260810 14:53:25.782829 2418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38771d48f92a49e5bd7716d58d6ef662" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44535 } }
I20260810 14:53:25.782873 2413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:25.783201 2419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38771d48f92a49e5bd7716d58d6ef662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38771d48f92a49e5bd7716d58d6ef662" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44535 } } }
I20260810 14:53:25.783313 2419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:25.783499 2420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38771d48f92a49e5bd7716d58d6ef662. Latest consensus state: current_term: 1 leader_uuid: "38771d48f92a49e5bd7716d58d6ef662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38771d48f92a49e5bd7716d58d6ef662" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44535 } } }
I20260810 14:53:25.783581 2420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:25.783934 2425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:25.784477 2425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:25.785971 2425 catalog_manager.cc:1347] Generated new cluster ID: b5bbcf63d79e4bb19c714ec4cd656f0f
I20260810 14:53:25.786029 2425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 14:53:25 dist-test-slave-0b58 krb5kdc[2324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:25.795152 2417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:25.782011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50296 (local address 127.30.74.126:44535)
0810 14:53:25.782192 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:25.782201 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:25.782238 (+ 37us) server_negotiation.cc:407] Connection header received
0810 14:53:25.782347 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:25.782373 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:25.782431 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:25.782576 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:25.783879 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:25.785263 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:25.788639 (+ 3376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:25.789236 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:25.789373 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:25.791715 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:25.791720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:25.791720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:25.791732 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:25.793480 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:25.793863 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:25.793866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:25.793866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:25.793961 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:25.794180 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:25.794181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:25.794181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:25.794438 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:25.794611 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:25.794723 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 14:53:25.794985 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":103,"threads_started":1}
I20260810 14:53:25.798110 2425 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:25.798789 2425 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:25.804505 2425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38771d48f92a49e5bd7716d58d6ef662: Generated new TSK 0
I20260810 14:53:25.804677 2425 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:53:25.835338 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:44535
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:25.919864 2439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:25.920121 2439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:25.920162 2439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:25.921404 2439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:25.921478 2439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:25.921525 2439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:53:25.923698 2439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:44535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:25.923965 2439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:25.924237 2439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:25.929248 2444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:25.929298 2448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:25.929445 2439 server_base.cc:1034] running on GCE node
W20260810 14:53:25.929253 2446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:25.929812 2439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:25.930028 2439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:25.931190 2439 hybrid_clock.cc:648] HybridClock initialized: now 1786373605931174 us; error 46 us; skew 500 ppm
Aug 10 14:53:25 dist-test-slave-0b58 krb5kdc[2324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373605, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:25.933785 2439 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:53:25.934237 2439 webserver.cc:460] Webserver started at http://127.30.74.65:43405/ using document root <none> and password file <none>
I20260810 14:53:25.934464 2439 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:25.934535 2439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:25.934698 2439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:25.935770 2439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "2e40357ad2604bf68eb2e25319ce19c6"
format_stamp: "Formatted at 2026-08-10 14:53:25 on dist-test-slave-0b58"
server_key: "bb20c6ff04224aac3a49e69f211ed3d1"
server_key_iv: "a885b7c9f8ff61f43329e2fa290425cc"
server_key_version: "encryptionkey@0"
I20260810 14:53:25.936146 2439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "2e40357ad2604bf68eb2e25319ce19c6"
format_stamp: "Formatted at 2026-08-10 14:53:25 on dist-test-slave-0b58"
server_key: "bb20c6ff04224aac3a49e69f211ed3d1"
server_key_iv: "a885b7c9f8ff61f43329e2fa290425cc"
server_key_version: "encryptionkey@0"
I20260810 14:53:25.937564 2439 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:25.938411 2456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:25.938607 2439 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:25.938728 2439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "2e40357ad2604bf68eb2e25319ce19c6"
format_stamp: "Formatted at 2026-08-10 14:53:25 on dist-test-slave-0b58"
server_key: "bb20c6ff04224aac3a49e69f211ed3d1"
server_key_iv: "a885b7c9f8ff61f43329e2fa290425cc"
server_key_version: "encryptionkey@0"
I20260810 14:53:25.938805 2439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:25.951040 2439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:25.953302 2439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:25.953474 2439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:25.953740 2439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:25.954064 2439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:25.954119 2439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:25.954154 2439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:25.954201 2439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:25.959978 2439 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:45937
I20260810 14:53:25.960090 2576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:45937 every 8 connection(s)
I20260810 14:53:25.960444 2439 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:53:25.961544 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 2439
I20260810 14:53:25.961650 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:53:25.961866 31017 external_mini_cluster.cc:1428] Setting key 910aecd52e0860861063ccb50b34f9fb
Aug 10 14:53:25 dist-test-slave-0b58 krb5kdc[2324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373605, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
I20260810 14:53:25.971454 2417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:25.961264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:44273 (local address 127.30.74.126:44535)
0810 14:53:25.961333 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:25.961335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:25.961509 (+ 174us) server_negotiation.cc:407] Connection header received
0810 14:53:25.962713 (+ 1204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:25.962714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:25.962727 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:25.962799 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:25.963385 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:25.964135 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:25.965205 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:25.965562 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:25.965640 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:25.968373 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:25.968374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:25.968377 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:25.968390 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:25.969901 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:25.970350 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:25.970352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:25.970352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:25.970421 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:25.970692 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:25.970693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:25.970694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:25.970826 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:25.970901 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:25.971260 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 14:53:25.971358 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 14:53:25.971645 2579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:25.961213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:44535 (local address 127.30.74.65:44273)
0810 14:53:25.961374 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:25.961416 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:25.962601 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:25.962854 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:25.962857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:25.963104 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:25.963293 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:25.963297 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:25.964252 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:25.964253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:25.965111 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:25.965113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:25.965676 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:25.965678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:25.965825 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:25.966442 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:25.966452 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:25.968203 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:25.970010 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:25.970018 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:25.970021 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:25.970265 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:25.970490 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:25.970491 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:25.970492 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:25.970621 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:25.970903 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:25.970912 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:25.971099 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 14:53:25.971222 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 14:53:25.971396 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260810 14:53:25.972131 2577 heartbeater.cc:347] Connected to a master server at 127.30.74.126:44535
I20260810 14:53:25.972234 2577 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:25.972457 2577 heartbeater.cc:511] Master 127.30.74.126:44535 requested a full tablet report, sending...
I20260810 14:53:25.973008 2372 ts_manager.cc:194] Registered new tserver with Master: 2e40357ad2604bf68eb2e25319ce19c6 (127.30.74.65:45937)
I20260810 14:53:25.973630 2372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:44273
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:53:25.989180 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:44535
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:26.072515 2582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:26.072759 2582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:26.072799 2582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:26.073952 2582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:26.074020 2582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:26.074067 2582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:53:26.075613 2582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:44535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:26.075888 2582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:26.076128 2582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:26.081058 2589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:26.081044 2591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:26.081045 2588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:26.081267 2582 server_base.cc:1034] running on GCE node
I20260810 14:53:26.081589 2582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:26.081853 2582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:26.083017 2582 hybrid_clock.cc:648] HybridClock initialized: now 1786373606083006 us; error 33 us; skew 500 ppm
Aug 10 14:53:26 dist-test-slave-0b58 krb5kdc[2324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373606, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:26.085814 2582 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:53:26.086324 2582 webserver.cc:460] Webserver started at http://127.30.74.66:33607/ using document root <none> and password file <none>
I20260810 14:53:26.086560 2582 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:26.086633 2582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:26.086784 2582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:26.087828 2582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "a775d0460f944c608d5fec01eacbc4d9"
format_stamp: "Formatted at 2026-08-10 14:53:26 on dist-test-slave-0b58"
server_key: "ca4fd34744e736af9802b282e5ab2411"
server_key_iv: "bcc7c18f5bc90c54a5970cbb6effd495"
server_key_version: "encryptionkey@0"
I20260810 14:53:26.088181 2582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "a775d0460f944c608d5fec01eacbc4d9"
format_stamp: "Formatted at 2026-08-10 14:53:26 on dist-test-slave-0b58"
server_key: "ca4fd34744e736af9802b282e5ab2411"
server_key_iv: "bcc7c18f5bc90c54a5970cbb6effd495"
server_key_version: "encryptionkey@0"
I20260810 14:53:26.089664 2582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:26.090544 2599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:26.090736 2582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 14:53:26.090852 2582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "a775d0460f944c608d5fec01eacbc4d9"
format_stamp: "Formatted at 2026-08-10 14:53:26 on dist-test-slave-0b58"
server_key: "ca4fd34744e736af9802b282e5ab2411"
server_key_iv: "bcc7c18f5bc90c54a5970cbb6effd495"
server_key_version: "encryptionkey@0"
I20260810 14:53:26.090942 2582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:26.113407 2582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:26.115686 2582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:26.115878 2582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:26.116142 2582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:26.116516 2582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:26.116580 2582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:26.116626 2582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:26.116663 2582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:26.122910 2582 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:45169
I20260810 14:53:26.122961 2716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:45169 every 8 connection(s)
I20260810 14:53:26.123302 2582 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:53:26.124878 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 2582
I20260810 14:53:26.124971 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:53:26.125155 31017 external_mini_cluster.cc:1428] Setting key e065f96d6ecd1c85b22898a8cf810e3b
Aug 10 14:53:26 dist-test-slave-0b58 krb5kdc[2324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373606, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:26.134382 2417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:26.124079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:59601 (local address 127.30.74.126:44535)
0810 14:53:26.124141 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:26.124143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:26.124446 (+ 303us) server_negotiation.cc:407] Connection header received
0810 14:53:26.125644 (+ 1198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:26.125645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:26.125654 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:26.125736 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:26.126661 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:26.127168 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:26.128472 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:26.128701 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:26.128774 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:26.131201 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:26.131203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:26.131203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:26.131213 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:26.132927 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:26.133361 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:26.133363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:26.133363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:26.133430 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:26.133675 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:26.133676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:26.133676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:26.133831 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:26.133913 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:26.134162 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 14:53:26.134270 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 14:53:26.134435 2719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:26.124045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:44535 (local address 127.30.74.66:59601)
0810 14:53:26.124248 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:26.124344 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:26.125543 (+ 1199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:26.125809 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:26.125813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:26.126268 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:26.126561 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:26.126567 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:26.127263 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:26.127264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:26.128383 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:26.128386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:26.128814 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:26.128817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:26.128940 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:26.129502 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:26.129511 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:26.131101 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:26.133023 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:26.133030 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:26.133032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:26.133261 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:26.133505 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:26.133506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:26.133506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:26.133607 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:26.133903 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:26.133908 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:26.134028 (+ 120us) client_negotiation.cc:769] Sending connection context
0810 14:53:26.134111 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 14:53:26.134238 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
I20260810 14:53:26.135121 2717 heartbeater.cc:347] Connected to a master server at 127.30.74.126:44535
I20260810 14:53:26.135238 2717 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:26.135437 2717 heartbeater.cc:511] Master 127.30.74.126:44535 requested a full tablet report, sending...
I20260810 14:53:26.135826 2372 ts_manager.cc:194] Registered new tserver with Master: a775d0460f944c608d5fec01eacbc4d9 (127.30.74.66:45169)
I20260810 14:53:26.136175 2372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:59601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:53:26.155534 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:44535
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:26.245334 2722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:26.245537 2722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:26.245561 2722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:26.246809 2722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:26.246855 2722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:26.246881 2722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:53:26.248611 2722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:44535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:26.248873 2722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:26.249126 2722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:26.254482 2731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:26.254472 2729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:26.254513 2722 server_base.cc:1034] running on GCE node
W20260810 14:53:26.254472 2728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:26.254871 2722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:26.255092 2722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:26.256340 2722 hybrid_clock.cc:648] HybridClock initialized: now 1786373606256326 us; error 32 us; skew 500 ppm
Aug 10 14:53:26 dist-test-slave-0b58 krb5kdc[2324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373606, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:26.258966 2722 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:53:26.259449 2722 webserver.cc:460] Webserver started at http://127.30.74.67:38357/ using document root <none> and password file <none>
I20260810 14:53:26.259709 2722 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:26.259781 2722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:26.259936 2722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:26.261076 2722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "1b95af6b9b9b460eb73d164631252795"
format_stamp: "Formatted at 2026-08-10 14:53:26 on dist-test-slave-0b58"
server_key: "b0832978907c64d1fb6f81cd0684f873"
server_key_iv: "6eef7ca03d80281e813e4347a5f04cf4"
server_key_version: "encryptionkey@0"
I20260810 14:53:26.261466 2722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "1b95af6b9b9b460eb73d164631252795"
format_stamp: "Formatted at 2026-08-10 14:53:26 on dist-test-slave-0b58"
server_key: "b0832978907c64d1fb6f81cd0684f873"
server_key_iv: "6eef7ca03d80281e813e4347a5f04cf4"
server_key_version: "encryptionkey@0"
I20260810 14:53:26.262848 2722 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 14:53:26.263608 2738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:26.263793 2722 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 14:53:26.263903 2722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "1b95af6b9b9b460eb73d164631252795"
format_stamp: "Formatted at 2026-08-10 14:53:26 on dist-test-slave-0b58"
server_key: "b0832978907c64d1fb6f81cd0684f873"
server_key_iv: "6eef7ca03d80281e813e4347a5f04cf4"
server_key_version: "encryptionkey@0"
I20260810 14:53:26.263980 2722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:26.277132 2722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:26.279551 2722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:26.279726 2722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:26.279971 2722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:26.280457 2722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:26.280515 2722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:26.280557 2722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:26.280586 2722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:26.287106 2722 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:45417
I20260810 14:53:26.287145 2856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:45417 every 8 connection(s)
I20260810 14:53:26.287488 2722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:53:26.291896 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 2722
I20260810 14:53:26.291997 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:53:26.292234 31017 external_mini_cluster.cc:1428] Setting key 9aa90352ba564efbd145abe72caed259
Aug 10 14:53:26 dist-test-slave-0b58 krb5kdc[2324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373606, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:26.297281 2417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:26.288279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:55775 (local address 127.30.74.126:44535)
0810 14:53:26.288341 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:26.288343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:26.288556 (+ 213us) server_negotiation.cc:407] Connection header received
0810 14:53:26.289495 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:26.289497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:26.289508 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:26.289561 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:26.290291 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:26.290784 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:26.292065 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:26.292365 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:26.292454 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:26.294792 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:26.294794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:26.294794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:26.294802 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:26.296066 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:26.296515 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:26.296515 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:26.296516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:26.296569 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:26.296792 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:26.296793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:26.296793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:26.296892 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:26.296938 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:26.297156 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 14:53:26.297215 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 14:53:26.297547 2859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:26.288214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:44535 (local address 127.30.74.67:55775)
0810 14:53:26.288432 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:26.288469 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:26.289400 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:26.289618 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:26.289621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:26.289993 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:26.290201 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:26.290207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:26.290868 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:26.290869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:26.291697 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:26.291699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:26.292469 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:26.292470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:26.292626 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:26.293224 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:26.293232 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:26.294693 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:26.296139 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:26.296147 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:26.296149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:26.296449 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:26.296631 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:26.296631 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:26.296632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:26.296750 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:26.296944 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:26.296948 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:26.297063 (+ 115us) client_negotiation.cc:769] Sending connection context
0810 14:53:26.297144 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 14:53:26.297268 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":141,"threads_started":1}
I20260810 14:53:26.298026 2857 heartbeater.cc:347] Connected to a master server at 127.30.74.126:44535
I20260810 14:53:26.298113 2857 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:26.298372 2857 heartbeater.cc:511] Master 127.30.74.126:44535 requested a full tablet report, sending...
I20260810 14:53:26.298740 2372 ts_manager.cc:194] Registered new tserver with Master: 1b95af6b9b9b460eb73d164631252795 (127.30.74.67:45417)
I20260810 14:53:26.299116 2372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:55775
I20260810 14:53:26.303982 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:53:26 dist-test-slave-0b58 krb5kdc[2324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 14:53:26 dist-test-slave-0b58 krb5kdc[2324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.65@KRBTEST.COM
I20260810 14:53:26.328080 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:26.320298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49660 (local address 127.30.74.65:45937)
0810 14:53:26.320476 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:26.320481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:26.320494 (+ 13us) server_negotiation.cc:407] Connection header received
0810 14:53:26.320521 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:26.320523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:26.320537 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:26.320607 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:26.321185 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:26.321913 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:26.322635 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:26.322985 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:26.323046 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:26.325274 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:26.325283 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:26.325283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:26.325290 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:26.326959 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:26.327284 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:26.327287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:26.327287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:26.327353 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:26.327589 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:26.327590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:26.327591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:26.327738 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:26.327847 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:26.327933 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 14:53:26.327998 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
Aug 10 14:53:26 dist-test-slave-0b58 krb5kdc[2324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:26.336480 2417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:26.329663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50308 (local address 127.30.74.126:44535)
0810 14:53:26.329758 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:26.329760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:26.329772 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:53:26.329817 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:26.329818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:26.329832 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:26.329893 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:26.330365 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:26.330850 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:26.331499 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:26.331676 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:26.331728 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:26.333863 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:26.333864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:26.333865 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:26.333871 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:26.335291 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:26.335585 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:26.335586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:26.335586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:26.335635 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:26.335914 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:26.335915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:26.335915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:26.336052 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:26.336125 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:26.336315 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 14:53:26.336378 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 14:53:26.336760 2372 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50308
I20260810 14:53:26.337157 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 2439
I20260810 14:53:26.344561 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 2582
I20260810 14:53:26.356153 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 2722
I20260810 14:53:26.361408 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 2337
2026-08-10T14:53:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1941 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:53:26 dist-test-slave-0b58 krb5kdc[2870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:53:26 dist-test-slave-0b58 krb5kdc[2870](info): set up 2 sockets
Aug 10 14:53:26 dist-test-slave-0b58 krb5kdc[2870](info): commencing operation
krb5kdc: starting...
W20260810 14:53:27.446811 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:53:27 dist-test-slave-0b58 krb5kdc[2870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:53:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:53:27.575469 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:33895
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:46177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:33895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:27.660557 2883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:27.660764 2883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:27.660787 2883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:27.661971 2883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:27.662014 2883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:27.662029 2883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:27.662041 2883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:27.662092 2883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:27.663606 2883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:46177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:33895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:33895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:27.663866 2883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:27.664101 2883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:27.668974 2888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:27.668974 2889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:27.669121 2883 server_base.cc:1034] running on GCE node
W20260810 14:53:27.669006 2891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:27.669453 2883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:27.669818 2883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:27.671005 2883 hybrid_clock.cc:648] HybridClock initialized: now 1786373607670992 us; error 46 us; skew 500 ppm
Aug 10 14:53:27 dist-test-slave-0b58 krb5kdc[2870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373607, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:27.673547 2883 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:53:27.673918 2883 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:27.673995 2883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:27.674146 2883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:27.675186 2883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "842b85f8ec0f4f1582d72ee3afebaa78"
format_stamp: "Formatted at 2026-08-10 14:53:27 on dist-test-slave-0b58"
server_key: "bdeebfeedee96fa501989ac15d398ea9"
server_key_iv: "4b50808c0fc13c1ecc7c4114bfd50fe7"
server_key_version: "encryptionkey@0"
I20260810 14:53:27.675561 2883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "842b85f8ec0f4f1582d72ee3afebaa78"
format_stamp: "Formatted at 2026-08-10 14:53:27 on dist-test-slave-0b58"
server_key: "bdeebfeedee96fa501989ac15d398ea9"
server_key_iv: "4b50808c0fc13c1ecc7c4114bfd50fe7"
server_key_version: "encryptionkey@0"
I20260810 14:53:27.676978 2883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:53:27.677695 2897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:27.677884 2883 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:27.677994 2883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "842b85f8ec0f4f1582d72ee3afebaa78"
format_stamp: "Formatted at 2026-08-10 14:53:27 on dist-test-slave-0b58"
server_key: "bdeebfeedee96fa501989ac15d398ea9"
server_key_iv: "4b50808c0fc13c1ecc7c4114bfd50fe7"
server_key_version: "encryptionkey@0"
I20260810 14:53:27.678073 2883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:27.696431 2883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:27.698292 2883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:27.698460 2883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:27.702443 2883 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:33895
I20260810 14:53:27.702491 2951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:33895 every 8 connection(s)
I20260810 14:53:27.702821 2883 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:27.703833 2952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:27.706156 2952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78: Bootstrap starting.
I20260810 14:53:27.706717 2952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:27.706956 2952 log.cc:824] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:27.707615 2952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78: No bootstrap required, opened a new log
I20260810 14:53:27.709177 2952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842b85f8ec0f4f1582d72ee3afebaa78" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 33895 } }
I20260810 14:53:27.709347 2952 raft_consensus.cc:374] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:27.709370 2952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842b85f8ec0f4f1582d72ee3afebaa78, State: Initialized, Role: FOLLOWER
I20260810 14:53:27.709525 2952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842b85f8ec0f4f1582d72ee3afebaa78" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 33895 } }
I20260810 14:53:27.709614 2952 raft_consensus.cc:388] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:27.709638 2952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:27.709666 2952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:27.710456 2952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842b85f8ec0f4f1582d72ee3afebaa78" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 33895 } }
I20260810 14:53:27.710544 2952 leader_election.cc:302] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 842b85f8ec0f4f1582d72ee3afebaa78; no voters:
I20260810 14:53:27.710670 2952 leader_election.cc:288] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:27.710758 2955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:27.710930 2955 raft_consensus.cc:686] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [term 1 LEADER]: Becoming Leader. State: Replica: 842b85f8ec0f4f1582d72ee3afebaa78, State: Running, Role: LEADER
I20260810 14:53:27.710997 2952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:27.711050 2955 consensus_queue.cc:237] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842b85f8ec0f4f1582d72ee3afebaa78" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 33895 } }
I20260810 14:53:27.711267 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 2883
I20260810 14:53:27.711404 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:27.711619 31017 external_mini_cluster.cc:1428] Setting key 97c495c4f4c3458f2bb2b0eb7713a483
I20260810 14:53:27.712080 2956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "842b85f8ec0f4f1582d72ee3afebaa78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842b85f8ec0f4f1582d72ee3afebaa78" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 33895 } } }
I20260810 14:53:27.712175 2956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:27.712399 2957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 842b85f8ec0f4f1582d72ee3afebaa78. Latest consensus state: current_term: 1 leader_uuid: "842b85f8ec0f4f1582d72ee3afebaa78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842b85f8ec0f4f1582d72ee3afebaa78" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 33895 } } }
I20260810 14:53:27.712471 2957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78 [sys.catalog]: This master's current role is: LEADER
W20260810 14:53:27.713042 2972 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:53:27.713104 2972 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:53:27.713147 2968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:27.713344 2968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:27.714980 2968 catalog_manager.cc:1347] Generated new cluster ID: 01e150ab565f4e30af441925eac623cd
I20260810 14:53:27.715037 2968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 14:53:27 dist-test-slave-0b58 krb5kdc[2870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:27.722625 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:27.712661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37342 (local address 127.30.74.126:33895)
0810 14:53:27.712847 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:27.712857 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:27.712906 (+ 49us) server_negotiation.cc:407] Connection header received
0810 14:53:27.713027 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:27.713032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:27.713077 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:27.713343 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:27.713984 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:27.715326 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:27.716108 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:27.716636 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:27.716768 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:27.719182 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:27.719186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:27.719186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:27.719202 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:27.720974 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:27.721380 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:27.721383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:27.721384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:27.721484 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:27.721700 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:27.721701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:27.721701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:27.721975 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:27.722151 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:27.722250 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 14:53:27.722411 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260810 14:53:27.724654 2968 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:27.725381 2968 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:27.729015 2968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78: Generated new TSK 0
I20260810 14:53:27.729202 2968 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:53:27.763955 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:33895
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:46177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:27.868059 2978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:27.868340 2978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:27.868396 2978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:27.869573 2978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:27.869644 2978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:27.869689 2978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:53:27.871229 2978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:46177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:33895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:27.871497 2978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:27.871737 2978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:27.877580 2988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:27.877611 2986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:27.877622 2985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:27.877935 2978 server_base.cc:1034] running on GCE node
I20260810 14:53:27.878145 2978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:27.878413 2978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:27.879570 2978 hybrid_clock.cc:648] HybridClock initialized: now 1786373607879548 us; error 39 us; skew 500 ppm
Aug 10 14:53:27 dist-test-slave-0b58 krb5kdc[2870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373607, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:27.882175 2978 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:53:27.882561 2978 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:27.882611 2978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:27.882706 2978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:27.883831 2978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "0efc802bbe194f53b03e4a8085d7a515"
format_stamp: "Formatted at 2026-08-10 14:53:27 on dist-test-slave-0b58"
server_key: "50a4da2ca43d545fbe40a07f34ee8585"
server_key_iv: "07f402b5c9594dcbe393fb6d8af351f2"
server_key_version: "encryptionkey@0"
I20260810 14:53:27.884155 2978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "0efc802bbe194f53b03e4a8085d7a515"
format_stamp: "Formatted at 2026-08-10 14:53:27 on dist-test-slave-0b58"
server_key: "50a4da2ca43d545fbe40a07f34ee8585"
server_key_iv: "07f402b5c9594dcbe393fb6d8af351f2"
server_key_version: "encryptionkey@0"
I20260810 14:53:27.885497 2978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:27.886263 2994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:27.886439 2978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:27.886524 2978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "0efc802bbe194f53b03e4a8085d7a515"
format_stamp: "Formatted at 2026-08-10 14:53:27 on dist-test-slave-0b58"
server_key: "50a4da2ca43d545fbe40a07f34ee8585"
server_key_iv: "07f402b5c9594dcbe393fb6d8af351f2"
server_key_version: "encryptionkey@0"
I20260810 14:53:27.886579 2978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:27.898281 2978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:27.900159 2978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:27.900345 2978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:27.900573 2978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:27.900882 2978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:27.900916 2978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:27.900939 2978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:27.900952 2978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:27.906639 2978 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:43427
I20260810 14:53:27.906764 3116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:43427 every 8 connection(s)
I20260810 14:53:27.906953 2978 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:53:27.910872 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 2978
I20260810 14:53:27.911006 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:53:27.911234 31017 external_mini_cluster.cc:1428] Setting key 7a8ef0068e177e75946a8a551ec4afaf
Aug 10 14:53:27 dist-test-slave-0b58 krb5kdc[2870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373607, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:27.917536 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:27.907851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:59387 (local address 127.30.74.126:33895)
0810 14:53:27.907925 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:27.907928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:27.908152 (+ 224us) server_negotiation.cc:407] Connection header received
0810 14:53:27.908780 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:27.908781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:27.908795 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:27.908853 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:27.909798 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:27.910315 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:27.911484 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:27.911750 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:27.911843 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:27.914441 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:27.914443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:27.914443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:27.914454 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:27.915927 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:27.916454 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:27.916455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:27.916456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:27.916530 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:27.916819 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:27.916820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:27.916821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:27.916955 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:27.917026 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:27.917347 (+ 321us) server_negotiation.cc:299] Negotiation successful
0810 14:53:27.917442 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 14:53:27.917755 3119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:27.907826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:33895 (local address 127.30.74.65:59387)
0810 14:53:27.908025 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:27.908062 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:27.908627 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:27.908928 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:27.908932 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:27.909210 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:27.909709 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:27.909714 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:27.910395 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:27.910395 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:27.911386 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:27.911389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:27.911867 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:27.911868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:27.912067 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:27.912668 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:27.912677 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:27.914325 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:27.916065 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:27.916072 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:27.916074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:27.916372 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:27.916602 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:27.916603 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:27.916603 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:27.916739 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:27.917032 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:27.917040 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:27.917179 (+ 139us) client_negotiation.cc:769] Sending connection context
0810 14:53:27.917309 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 14:53:27.917479 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":108,"threads_started":1}
I20260810 14:53:27.918366 3117 heartbeater.cc:347] Connected to a master server at 127.30.74.126:33895
I20260810 14:53:27.918463 3117 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:27.918642 3117 heartbeater.cc:511] Master 127.30.74.126:33895 requested a full tablet report, sending...
I20260810 14:53:27.919278 2914 ts_manager.cc:194] Registered new tserver with Master: 0efc802bbe194f53b03e4a8085d7a515 (127.30.74.65:43427)
I20260810 14:53:27.920303 2914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:59387
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:53:27.938697 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:33895
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:46177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:28.022246 3122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:28.022478 3122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:28.022504 3122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:28.023638 3122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:28.023674 3122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:28.023700 3122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:53:28.025285 3122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:46177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:33895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:28.025487 3122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:28.025674 3122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:28.030897 3127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:28.030929 3128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:28.030897 3130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:28.031016 3122 server_base.cc:1034] running on GCE node
I20260810 14:53:28.031440 3122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:28.031695 3122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:28.032920 3122 hybrid_clock.cc:648] HybridClock initialized: now 1786373608032908 us; error 46 us; skew 500 ppm
Aug 10 14:53:28 dist-test-slave-0b58 krb5kdc[2870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373608, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:28.035705 3122 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:53:28.036018 3122 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:28.036062 3122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:28.036164 3122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:28.037271 3122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "80293567cec5462182e54712356ce036"
format_stamp: "Formatted at 2026-08-10 14:53:28 on dist-test-slave-0b58"
server_key: "bb0156b04036bcabf8dd2e208ddb64d7"
server_key_iv: "adbb739ca80ce15f2fded3786400da41"
server_key_version: "encryptionkey@0"
I20260810 14:53:28.037585 3122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "80293567cec5462182e54712356ce036"
format_stamp: "Formatted at 2026-08-10 14:53:28 on dist-test-slave-0b58"
server_key: "bb0156b04036bcabf8dd2e208ddb64d7"
server_key_iv: "adbb739ca80ce15f2fded3786400da41"
server_key_version: "encryptionkey@0"
I20260810 14:53:28.038818 3122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 14:53:28.039546 3136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:28.039702 3122 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 14:53:28.039789 3122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "80293567cec5462182e54712356ce036"
format_stamp: "Formatted at 2026-08-10 14:53:28 on dist-test-slave-0b58"
server_key: "bb0156b04036bcabf8dd2e208ddb64d7"
server_key_iv: "adbb739ca80ce15f2fded3786400da41"
server_key_version: "encryptionkey@0"
I20260810 14:53:28.039844 3122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:28.051949 3122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:28.054076 3122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:28.054225 3122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:28.054417 3122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:28.054749 3122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:28.054785 3122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:28.054806 3122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:28.054854 3122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:28.060881 3122 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:38521
I20260810 14:53:28.060957 3257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:38521 every 8 connection(s)
I20260810 14:53:28.061246 3122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:53:28.064052 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 3122
I20260810 14:53:28.064188 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:53:28.064399 31017 external_mini_cluster.cc:1428] Setting key 912b7c9a6a1c9681d2f7040aa7f14efd
Aug 10 14:53:28 dist-test-slave-0b58 krb5kdc[2870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373608, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:28.071652 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.062035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:48689 (local address 127.30.74.126:33895)
0810 14:53:28.062099 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:28.062102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:28.062297 (+ 195us) server_negotiation.cc:407] Connection header received
0810 14:53:28.062853 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:28.062854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:28.062869 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:28.062927 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:28.063924 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.064576 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.065748 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.065955 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.066033 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:28.068566 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:28.068568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:28.068568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:28.068576 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:28.070204 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:28.070627 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:28.070629 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:28.070629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:28.070692 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:28.070952 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:28.070953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:28.070954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:28.071079 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:28.071173 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:28.071466 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 14:53:28.071562 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 14:53:28.071871 3261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.061992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:33895 (local address 127.30.74.66:48689)
0810 14:53:28.062141 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:28.062186 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:28.062741 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:28.062993 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:28.062996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:28.063279 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:28.063792 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:28.063797 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.064684 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.064685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:28.065633 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:28.065636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.066041 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.066042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:28.066191 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:28.066749 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:28.066758 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:28.068462 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:28.070298 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:28.070305 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:28.070308 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:28.070546 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:28.070769 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:28.070770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:28.070770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:28.070884 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:28.071167 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:28.071174 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:28.071323 (+ 149us) client_negotiation.cc:769] Sending connection context
0810 14:53:28.071429 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 14:53:28.071597 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260810 14:53:28.072381 3259 heartbeater.cc:347] Connected to a master server at 127.30.74.126:33895
I20260810 14:53:28.072476 3259 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:28.072656 3259 heartbeater.cc:511] Master 127.30.74.126:33895 requested a full tablet report, sending...
I20260810 14:53:28.073112 2914 ts_manager.cc:194] Registered new tserver with Master: 80293567cec5462182e54712356ce036 (127.30.74.66:38521)
I20260810 14:53:28.073510 2914 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:48689
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:53:28.091588 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:33895
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:46177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:28.175909 3264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:28.176126 3264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:28.176153 3264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:28.177423 3264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:28.177470 3264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:28.177500 3264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:53:28.178983 3264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:46177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:33895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:28.179181 3264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:28.179366 3264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:28.184499 3273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:28.184499 3269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:28.184548 3270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:28.184819 3264 server_base.cc:1034] running on GCE node
I20260810 14:53:28.185043 3264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:28.185245 3264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:28.186408 3264 hybrid_clock.cc:648] HybridClock initialized: now 1786373608186384 us; error 48 us; skew 500 ppm
Aug 10 14:53:28 dist-test-slave-0b58 krb5kdc[2870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373608, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:28.188908 3264 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:53:28.189299 3264 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:28.189397 3264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:28.189575 3264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:28.190670 3264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "77a817711bd94bfb96b9a604450d3d62"
format_stamp: "Formatted at 2026-08-10 14:53:28 on dist-test-slave-0b58"
server_key: "07b88588f1091b100c7600661494fb12"
server_key_iv: "3eb960db75c0883f160876f4c10bb527"
server_key_version: "encryptionkey@0"
I20260810 14:53:28.190984 3264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "77a817711bd94bfb96b9a604450d3d62"
format_stamp: "Formatted at 2026-08-10 14:53:28 on dist-test-slave-0b58"
server_key: "07b88588f1091b100c7600661494fb12"
server_key_iv: "3eb960db75c0883f160876f4c10bb527"
server_key_version: "encryptionkey@0"
I20260810 14:53:28.192341 3264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 14:53:28.193126 3280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:28.193308 3264 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:28.193388 3264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "77a817711bd94bfb96b9a604450d3d62"
format_stamp: "Formatted at 2026-08-10 14:53:28 on dist-test-slave-0b58"
server_key: "07b88588f1091b100c7600661494fb12"
server_key_iv: "3eb960db75c0883f160876f4c10bb527"
server_key_version: "encryptionkey@0"
I20260810 14:53:28.193447 3264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:28.207099 3264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:28.209270 3264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:28.209389 3264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:28.209587 3264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:28.209859 3264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:28.209889 3264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:28.209911 3264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:28.209923 3264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:28.216336 3264 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:34939
I20260810 14:53:28.216354 3398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:34939 every 8 connection(s)
I20260810 14:53:28.216665 3264 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
Aug 10 14:53:28 dist-test-slave-0b58 krb5kdc[2870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373608, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:28.226579 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.217422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:47309 (local address 127.30.74.126:33895)
0810 14:53:28.217497 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:28.217500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:28.217664 (+ 164us) server_negotiation.cc:407] Connection header received
0810 14:53:28.218180 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:28.218181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:28.218194 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:28.218270 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:28.219273 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.219785 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.220901 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.221093 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.221156 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:28.223585 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:28.223587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:28.223587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:28.223596 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:28.225126 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:28.225562 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:28.225564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:28.225565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:28.225635 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:28.225890 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:28.225891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:28.225892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:28.226017 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:28.226104 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:28.226397 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 14:53:28.226478 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 14:53:28.226788 3401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.217346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:33895 (local address 127.30.74.67:47309)
0810 14:53:28.217533 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:28.217576 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:28.218083 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:28.218362 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:28.218365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:28.218642 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:28.219145 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:28.219151 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.219878 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.219878 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:28.220791 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:28.220794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.221174 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.221175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:28.221301 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:28.221795 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:28.221803 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:28.223480 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:28.225232 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:28.225240 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:28.225243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:28.225470 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:28.225704 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:28.225705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:28.225706 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:28.225822 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:28.226087 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:28.226104 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:28.226255 (+ 151us) client_negotiation.cc:769] Sending connection context
0810 14:53:28.226358 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 14:53:28.226531 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260810 14:53:28.227571 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 3264
I20260810 14:53:28.227674 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:53:28.227885 31017 external_mini_cluster.cc:1428] Setting key 2d92afa2db23313a265c2a4c3ebed138
I20260810 14:53:28.228523 3399 heartbeater.cc:347] Connected to a master server at 127.30.74.126:33895
I20260810 14:53:28.228636 3399 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:28.228822 3399 heartbeater.cc:511] Master 127.30.74.126:33895 requested a full tablet report, sending...
I20260810 14:53:28.229408 2913 ts_manager.cc:194] Registered new tserver with Master: 77a817711bd94bfb96b9a604450d3d62 (127.30.74.67:34939)
I20260810 14:53:28.229830 2913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:47309
I20260810 14:53:28.240231 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:53:28.247642 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.241774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37352 (local address 127.30.74.126:33895)
0810 14:53:28.241829 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:28.241831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:28.241840 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:28.241887 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:28.241888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:28.241898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:28.241963 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:28.242595 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.243123 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.243917 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.244120 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.244182 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:28.245378 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:28.245379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:28.245379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:28.245386 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:28.246617 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:28.246930 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:28.246931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:28.246931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:28.246981 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:28.247177 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:28.247178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:28.247178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:28.247340 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:28.247424 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:28.247513 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 14:53:28.247575 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:53:28.249620 2913 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:53:28.251186 2913 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:28.257915 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.253978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53986 (local address 127.30.74.65:43427)
0810 14:53:28.254173 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:28.254178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:28.254191 (+ 13us) server_negotiation.cc:407] Connection header received
0810 14:53:28.254257 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:28.254259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:28.254273 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:28.254346 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:28.254592 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.255384 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.256843 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.257462 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.257553 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:28.257615 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:28.257792 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 14:53:28.257832 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":94,"threads_started":1}
I20260810 14:53:28.257903 3414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.253835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:43427 (local address 127.0.0.1:53986)
0810 14:53:28.254132 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:28.254142 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:28.254184 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:28.254438 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:28.254438 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:28.254441 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:28.254509 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:28.254512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.255478 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.255479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:28.256753 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:28.256756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.257555 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.257556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:28.257650 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:28.257665 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 14:53:28.257769 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 14:53:28.257828 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":5,"thread_start_us":74,"threads_started":1}
I20260810 14:53:28.258193 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.253591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:38521 (local address 127.0.0.1:47610)
0810 14:53:28.253740 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:28.253775 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:28.253849 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:28.254213 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:28.254217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:28.254219 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:28.254515 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:28.254518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.255862 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.255863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:28.256941 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:28.256943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.257941 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.257943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:28.258010 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:28.258014 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:28.258071 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 14:53:28.258117 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":29,"thread_start_us":83,"threads_started":1}
I20260810 14:53:28.258495 3411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.253740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47610 (local address 127.30.74.66:38521)
0810 14:53:28.253940 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:28.253945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:28.253956 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:53:28.254002 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:28.254004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:28.254022 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:28.254106 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:28.254602 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.255739 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.257039 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.257766 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.257844 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:28.257900 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:28.258352 (+ 452us) server_negotiation.cc:299] Negotiation successful
0810 14:53:28.258395 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":85,"threads_started":1}
I20260810 14:53:28.258744 3412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.254006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:34939 (local address 127.0.0.1:47634)
0810 14:53:28.254437 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:28.254448 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:28.254505 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:28.254735 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:28.254736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:28.254738 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:28.254772 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:28.254773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.255994 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.255996 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:28.257023 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:28.257026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.257943 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.257944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:28.258400 (+ 456us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:28.258405 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:53:28.258463 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 14:53:28.258499 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":77,"threads_started":1}
I20260810 14:53:28.258683 3045 tablet_service.cc:1467] Processing CreateTablet for tablet 13e58c1cfec847c6a92f51ff2214a2c4 (DEFAULT_TABLE table=test-table [id=0e6dbf2b6eb24f09b370b7733d87a0bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:28.259006 3415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.254116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47634 (local address 127.30.74.67:34939)
0810 14:53:28.254351 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:28.254355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:28.254570 (+ 215us) server_negotiation.cc:407] Connection header received
0810 14:53:28.254623 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:28.254625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:28.254642 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:28.254722 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:28.254836 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.255885 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.257101 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.257797 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.257857 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:28.257907 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:28.258483 (+ 576us) server_negotiation.cc:299] Negotiation successful
0810 14:53:28.258920 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":120,"threads_started":1}
I20260810 14:53:28.259087 3045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13e58c1cfec847c6a92f51ff2214a2c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:28.259167 3188 tablet_service.cc:1467] Processing CreateTablet for tablet 13e58c1cfec847c6a92f51ff2214a2c4 (DEFAULT_TABLE table=test-table [id=0e6dbf2b6eb24f09b370b7733d87a0bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:28.259510 3188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13e58c1cfec847c6a92f51ff2214a2c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:28.260979 3331 tablet_service.cc:1467] Processing CreateTablet for tablet 13e58c1cfec847c6a92f51ff2214a2c4 (DEFAULT_TABLE table=test-table [id=0e6dbf2b6eb24f09b370b7733d87a0bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:28.261310 3331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13e58c1cfec847c6a92f51ff2214a2c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:28.262392 3416 tablet_bootstrap.cc:492] T 13e58c1cfec847c6a92f51ff2214a2c4 P 0efc802bbe194f53b03e4a8085d7a515: Bootstrap starting.
I20260810 14:53:28.262609 3418 tablet_bootstrap.cc:492] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036: Bootstrap starting.
I20260810 14:53:28.263217 3418 tablet_bootstrap.cc:654] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:28.263217 3416 tablet_bootstrap.cc:654] T 13e58c1cfec847c6a92f51ff2214a2c4 P 0efc802bbe194f53b03e4a8085d7a515: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:28.263900 3416 log.cc:824] T 13e58c1cfec847c6a92f51ff2214a2c4 P 0efc802bbe194f53b03e4a8085d7a515: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:28.263900 3418 log.cc:824] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:28.264395 3421 tablet_bootstrap.cc:492] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62: Bootstrap starting.
I20260810 14:53:28.265033 3418 tablet_bootstrap.cc:492] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036: No bootstrap required, opened a new log
I20260810 14:53:28.265034 3421 tablet_bootstrap.cc:654] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:28.265120 3418 ts_tablet_manager.cc:1397] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 14:53:28.265414 3416 tablet_bootstrap.cc:492] T 13e58c1cfec847c6a92f51ff2214a2c4 P 0efc802bbe194f53b03e4a8085d7a515: No bootstrap required, opened a new log
I20260810 14:53:28.265501 3416 ts_tablet_manager.cc:1397] T 13e58c1cfec847c6a92f51ff2214a2c4 P 0efc802bbe194f53b03e4a8085d7a515: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 14:53:28.265658 3421 log.cc:824] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:28.266476 3421 tablet_bootstrap.cc:492] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62: No bootstrap required, opened a new log
I20260810 14:53:28.266546 3421 ts_tablet_manager.cc:1397] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 14:53:28.266947 3418 raft_consensus.cc:348] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80293567cec5462182e54712356ce036" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38521 } } peers { permanent_uuid: "0efc802bbe194f53b03e4a8085d7a515" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43427 } } peers { permanent_uuid: "77a817711bd94bfb96b9a604450d3d62" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 34939 } }
I20260810 14:53:28.267098 3418 raft_consensus.cc:374] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:28.267143 3418 raft_consensus.cc:729] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80293567cec5462182e54712356ce036, State: Initialized, Role: FOLLOWER
I20260810 14:53:28.267258 3418 consensus_queue.cc:260] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80293567cec5462182e54712356ce036" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38521 } } peers { permanent_uuid: "0efc802bbe194f53b03e4a8085d7a515" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43427 } } peers { permanent_uuid: "77a817711bd94bfb96b9a604450d3d62" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 34939 } }
I20260810 14:53:28.267539 3418 ts_tablet_manager.cc:1428] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 14:53:28.267547 3416 raft_consensus.cc:348] T 13e58c1cfec847c6a92f51ff2214a2c4 P 0efc802bbe194f53b03e4a8085d7a515 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80293567cec5462182e54712356ce036" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38521 } } peers { permanent_uuid: "0efc802bbe194f53b03e4a8085d7a515" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43427 } } peers { permanent_uuid: "77a817711bd94bfb96b9a604450d3d62" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 34939 } }
I20260810 14:53:28.267674 3259 heartbeater.cc:503] Master 127.30.74.126:33895 was elected leader, sending a full tablet report...
I20260810 14:53:28.267715 3416 raft_consensus.cc:374] T 13e58c1cfec847c6a92f51ff2214a2c4 P 0efc802bbe194f53b03e4a8085d7a515 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:28.267757 3416 raft_consensus.cc:729] T 13e58c1cfec847c6a92f51ff2214a2c4 P 0efc802bbe194f53b03e4a8085d7a515 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0efc802bbe194f53b03e4a8085d7a515, State: Initialized, Role: FOLLOWER
I20260810 14:53:28.267872 3416 consensus_queue.cc:260] T 13e58c1cfec847c6a92f51ff2214a2c4 P 0efc802bbe194f53b03e4a8085d7a515 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80293567cec5462182e54712356ce036" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38521 } } peers { permanent_uuid: "0efc802bbe194f53b03e4a8085d7a515" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43427 } } peers { permanent_uuid: "77a817711bd94bfb96b9a604450d3d62" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 34939 } }
I20260810 14:53:28.268136 3416 ts_tablet_manager.cc:1428] T 13e58c1cfec847c6a92f51ff2214a2c4 P 0efc802bbe194f53b03e4a8085d7a515: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:53:28.268157 3117 heartbeater.cc:503] Master 127.30.74.126:33895 was elected leader, sending a full tablet report...
I20260810 14:53:28.268580 3421 raft_consensus.cc:348] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80293567cec5462182e54712356ce036" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38521 } } peers { permanent_uuid: "0efc802bbe194f53b03e4a8085d7a515" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43427 } } peers { permanent_uuid: "77a817711bd94bfb96b9a604450d3d62" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 34939 } }
I20260810 14:53:28.268759 3421 raft_consensus.cc:374] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:28.268803 3421 raft_consensus.cc:729] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77a817711bd94bfb96b9a604450d3d62, State: Initialized, Role: FOLLOWER
I20260810 14:53:28.268941 3421 consensus_queue.cc:260] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80293567cec5462182e54712356ce036" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38521 } } peers { permanent_uuid: "0efc802bbe194f53b03e4a8085d7a515" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43427 } } peers { permanent_uuid: "77a817711bd94bfb96b9a604450d3d62" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 34939 } }
I20260810 14:53:28.269259 3421 ts_tablet_manager.cc:1428] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 14:53:28.269302 3399 heartbeater.cc:503] Master 127.30.74.126:33895 was elected leader, sending a full tablet report...
W20260810 14:53:28.311883 3260 tablet.cc:2369] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:53:28.407819 3118 tablet.cc:2369] T 13e58c1cfec847c6a92f51ff2214a2c4 P 0efc802bbe194f53b03e4a8085d7a515: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:53:28.467252 3400 tablet.cc:2369] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:53:28.476806 3423 raft_consensus.cc:482] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:28.476925 3423 raft_consensus.cc:504] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80293567cec5462182e54712356ce036" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38521 } } peers { permanent_uuid: "0efc802bbe194f53b03e4a8085d7a515" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43427 } } peers { permanent_uuid: "77a817711bd94bfb96b9a604450d3d62" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 34939 } }
I20260810 14:53:28.477267 3423 leader_election.cc:288] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0efc802bbe194f53b03e4a8085d7a515 (127.30.74.65:43427), 77a817711bd94bfb96b9a604450d3d62 (127.30.74.67:34939)
I20260810 14:53:28.480406 3413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.477535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:56645 (local address 127.30.74.65:43427)
0810 14:53:28.477659 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:28.477661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:28.477672 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:53:28.477714 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:28.477715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:28.477726 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:28.477797 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:28.477973 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.478510 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.479508 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.480057 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.480126 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:28.480138 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:28.480291 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 14:53:28.480328 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 14:53:28.480413 3261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.477445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:43427 (local address 127.30.74.66:56645)
0810 14:53:28.477526 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:28.477541 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:28.477599 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:28.477833 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:28.477834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:28.477835 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:28.477899 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:28.477902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.478586 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.478587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:28.479421 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:28.479423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.480121 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.480122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:28.480193 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:28.480197 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:28.480295 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 14:53:28.480337 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 14:53:28.480578 3426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.477563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:34939 (local address 127.30.74.66:48531)
0810 14:53:28.477736 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:28.477746 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:28.477803 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:28.477968 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:28.477969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:28.477970 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:28.478040 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:28.478042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.478721 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.478721 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:28.479456 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:28.479477 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.480298 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.480299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:28.480377 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:28.480381 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:28.480443 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 14:53:28.480487 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":105,"threads_started":1}
I20260810 14:53:28.480584 3415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.477636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:48531 (local address 127.30.74.67:34939)
0810 14:53:28.477683 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:28.477685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:28.477811 (+ 126us) server_negotiation.cc:407] Connection header received
0810 14:53:28.477862 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:28.477863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:28.477873 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:28.477937 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:28.478105 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.478637 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.479610 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.480198 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.480303 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:28.480311 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:28.480459 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 14:53:28.480499 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:53:28.480777 3065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13e58c1cfec847c6a92f51ff2214a2c4" candidate_uuid: "80293567cec5462182e54712356ce036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0efc802bbe194f53b03e4a8085d7a515" is_pre_election: true
I20260810 14:53:28.480949 3065 raft_consensus.cc:2454] T 13e58c1cfec847c6a92f51ff2214a2c4 P 0efc802bbe194f53b03e4a8085d7a515 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80293567cec5462182e54712356ce036 in term 0.
I20260810 14:53:28.481163 3141 leader_election.cc:302] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0efc802bbe194f53b03e4a8085d7a515, 80293567cec5462182e54712356ce036; no voters:
I20260810 14:53:28.481276 3423 raft_consensus.cc:2781] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:28.481345 3423 raft_consensus.cc:482] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:28.481371 3423 raft_consensus.cc:3037] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:28.481581 3353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13e58c1cfec847c6a92f51ff2214a2c4" candidate_uuid: "80293567cec5462182e54712356ce036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77a817711bd94bfb96b9a604450d3d62" is_pre_election: true
I20260810 14:53:28.481725 3353 raft_consensus.cc:2454] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80293567cec5462182e54712356ce036 in term 0.
I20260810 14:53:28.482255 3423 raft_consensus.cc:504] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80293567cec5462182e54712356ce036" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38521 } } peers { permanent_uuid: "0efc802bbe194f53b03e4a8085d7a515" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43427 } } peers { permanent_uuid: "77a817711bd94bfb96b9a604450d3d62" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 34939 } }
I20260810 14:53:28.482390 3423 leader_election.cc:288] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [CANDIDATE]: Term 1 election: Requested vote from peers 0efc802bbe194f53b03e4a8085d7a515 (127.30.74.65:43427), 77a817711bd94bfb96b9a604450d3d62 (127.30.74.67:34939)
I20260810 14:53:28.482589 3353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13e58c1cfec847c6a92f51ff2214a2c4" candidate_uuid: "80293567cec5462182e54712356ce036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77a817711bd94bfb96b9a604450d3d62"
I20260810 14:53:28.482599 3065 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13e58c1cfec847c6a92f51ff2214a2c4" candidate_uuid: "80293567cec5462182e54712356ce036" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0efc802bbe194f53b03e4a8085d7a515"
I20260810 14:53:28.482658 3065 raft_consensus.cc:3037] T 13e58c1cfec847c6a92f51ff2214a2c4 P 0efc802bbe194f53b03e4a8085d7a515 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:28.482676 3353 raft_consensus.cc:3037] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:28.483345 3065 raft_consensus.cc:2454] T 13e58c1cfec847c6a92f51ff2214a2c4 P 0efc802bbe194f53b03e4a8085d7a515 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80293567cec5462182e54712356ce036 in term 1.
I20260810 14:53:28.483467 3353 raft_consensus.cc:2454] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80293567cec5462182e54712356ce036 in term 1.
I20260810 14:53:28.483515 3141 leader_election.cc:302] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0efc802bbe194f53b03e4a8085d7a515, 80293567cec5462182e54712356ce036; no voters:
I20260810 14:53:28.483686 3423 raft_consensus.cc:2781] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:28.483867 3423 raft_consensus.cc:686] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [term 1 LEADER]: Becoming Leader. State: Replica: 80293567cec5462182e54712356ce036, State: Running, Role: LEADER
I20260810 14:53:28.483978 3423 consensus_queue.cc:237] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80293567cec5462182e54712356ce036" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38521 } } peers { permanent_uuid: "0efc802bbe194f53b03e4a8085d7a515" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43427 } } peers { permanent_uuid: "77a817711bd94bfb96b9a604450d3d62" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 34939 } }
I20260810 14:53:28.484797 2913 catalog_manager.cc:5495] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80293567cec5462182e54712356ce036 (127.30.74.66). New cstate: current_term: 1 leader_uuid: "80293567cec5462182e54712356ce036" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80293567cec5462182e54712356ce036" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0efc802bbe194f53b03e4a8085d7a515" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77a817711bd94bfb96b9a604450d3d62" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 34939 } health_report { overall_health: UNKNOWN } } }
I20260810 14:53:28.540925 3411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.538466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47614 (local address 127.30.74.66:38521)
0810 14:53:28.538530 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:28.538531 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:28.538541 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:28.538578 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:28.538579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:28.538597 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:28.538668 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:28.538809 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.539281 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.540081 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:28.540319 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:28.540376 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:28.540533 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:28.540646 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:28.540727 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:28.540741 (+ 14us) server_negotiation.cc:299] Negotiation successful
0810 14:53:28.540776 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 14:53:28.542893 3065 raft_consensus.cc:1261] T 13e58c1cfec847c6a92f51ff2214a2c4 P 0efc802bbe194f53b03e4a8085d7a515 [term 1 FOLLOWER]: Refusing update from remote peer 80293567cec5462182e54712356ce036: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:28.542893 3353 raft_consensus.cc:1261] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62 [term 1 FOLLOWER]: Refusing update from remote peer 80293567cec5462182e54712356ce036: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:28.543282 3427 consensus_queue.cc:1035] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77a817711bd94bfb96b9a604450d3d62" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 34939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:28.543457 3423 consensus_queue.cc:1035] T 13e58c1cfec847c6a92f51ff2214a2c4 P 80293567cec5462182e54712356ce036 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0efc802bbe194f53b03e4a8085d7a515" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:28.545751 3430 mvcc.cc:205] Tried to move back new op lower bound from 7316986300588187648 to 7316986300350853120. Current Snapshot: MvccSnapshot[applied={T|T < 7316986300588187648}]
I20260810 14:53:28.546406 3432 mvcc.cc:205] Tried to move back new op lower bound from 7316986300588187648 to 7316986300350853120. Current Snapshot: MvccSnapshot[applied={T|T < 7316986300588187648}]
I20260810 14:53:28.546531 3431 mvcc.cc:205] Tried to move back new op lower bound from 7316986300588187648 to 7316986300350853120. Current Snapshot: MvccSnapshot[applied={T|T < 7316986300588187648 or (T in {7316986300588187648})}]
I20260810 14:53:28.548246 2914 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:28.548373 2914 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:28.549154 2914 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 842b85f8ec0f4f1582d72ee3afebaa78: Sending DeleteTablet for 3 replicas of tablet 13e58c1cfec847c6a92f51ff2214a2c4
I20260810 14:53:28.549489 3188 tablet_service.cc:1514] Processing DeleteTablet for tablet 13e58c1cfec847c6a92f51ff2214a2c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:28 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:47610
I20260810 14:53:28.549635 3045 tablet_service.cc:1514] Processing DeleteTablet for tablet 13e58c1cfec847c6a92f51ff2214a2c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:28 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:53986
I20260810 14:53:28.549726 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 2978
I20260810 14:53:28.549818 3331 tablet_service.cc:1514] Processing DeleteTablet for tablet 13e58c1cfec847c6a92f51ff2214a2c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:28 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:47634
I20260810 14:53:28.551990 3443 tablet_replica.cc:330] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62: stopping tablet replica
I20260810 14:53:28.552127 3443 raft_consensus.cc:2229] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:28.552245 3443 raft_consensus.cc:2258] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:28.552742 3443 ts_tablet_manager.cc:1905] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:28.554391 3443 ts_tablet_manager.cc:1918] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:28.554487 3443 log.cc:1196] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/13e58c1cfec847c6a92f51ff2214a2c4
I20260810 14:53:28.554786 3443 ts_tablet_manager.cc:1939] T 13e58c1cfec847c6a92f51ff2214a2c4 P 77a817711bd94bfb96b9a604450d3d62: Deleting consensus metadata
I20260810 14:53:28.555294 2901 catalog_manager.cc:4841] TS 77a817711bd94bfb96b9a604450d3d62 (127.30.74.67:34939): tablet 13e58c1cfec847c6a92f51ff2214a2c4 (table test-table [id=0e6dbf2b6eb24f09b370b7733d87a0bc]) successfully deleted
W20260810 14:53:28.559732 2901 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:43427 (error 108)
I20260810 14:53:28.560321 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 3122
I20260810 14:53:28.560503 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.560373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:43427 (local address 127.0.0.1:53988)
0810 14:53:28.560420 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:28.560463 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260810 14:53:28.562965 2901 catalog_manager.cc:4584] TS 0efc802bbe194f53b03e4a8085d7a515 (127.30.74.65:43427): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 13e58c1cfec847c6a92f51ff2214a2c4: Network error: Client connection negotiation failed: client connection to 127.30.74.65:43427: connect: Connection refused (error 111)
I20260810 14:53:28.567770 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.567616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:38521 (local address 127.0.0.1:47618)
0810 14:53:28.567690 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:28.567728 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.66:38521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 14:53:28.567780 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 3264
I20260810 14:53:28.574541 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 2883
2026-08-10T14:53:28Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2214 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T14:53:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:28Z Disabled control of system clock
I20260810 14:53:28.605667 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:42777
--webserver_interface=127.30.74.126
--webserver_port=0
--builtin_ntp_servers=127.30.74.84:33993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:42777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 14:53:28.688901 3448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:28.689121 3448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:28.689141 3448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:28.690235 3448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:28.690271 3448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:28.690284 3448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:28.690297 3448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:28.690310 3448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:28.691589 3448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:42777
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:42777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:28.691810 3448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:28.692054 3448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:28.696915 3453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:28.696895 3457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:28.697184 3454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:28.697162 3448 server_base.cc:1034] running on GCE node
I20260810 14:53:28.697427 3448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:28.697685 3448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:28.698834 3448 hybrid_clock.cc:648] HybridClock initialized: now 1786373608698820 us; error 30 us; skew 500 ppm
I20260810 14:53:28.700250 3448 webserver.cc:460] Webserver started at http://127.30.74.126:39555/ using document root <none> and password file <none>
I20260810 14:53:28.700578 3448 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:28.700652 3448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:28.700803 3448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:28.701932 3448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "7ffe77f95fb74174b1c4cfcb2163a8a3"
format_stamp: "Formatted at 2026-08-10 14:53:28 on dist-test-slave-0b58"
server_key: "87fef04878863f81c8dda2e66ebd5009"
server_key_iv: "4f1b5b8eacc1613921c87c2c848059ae"
server_key_version: "encryptionkey@0"
I20260810 14:53:28.702324 3448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "7ffe77f95fb74174b1c4cfcb2163a8a3"
format_stamp: "Formatted at 2026-08-10 14:53:28 on dist-test-slave-0b58"
server_key: "87fef04878863f81c8dda2e66ebd5009"
server_key_iv: "4f1b5b8eacc1613921c87c2c848059ae"
server_key_version: "encryptionkey@0"
I20260810 14:53:28.703729 3448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:53:28.704624 3463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:28.704797 3448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:28.704909 3448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "7ffe77f95fb74174b1c4cfcb2163a8a3"
format_stamp: "Formatted at 2026-08-10 14:53:28 on dist-test-slave-0b58"
server_key: "87fef04878863f81c8dda2e66ebd5009"
server_key_iv: "4f1b5b8eacc1613921c87c2c848059ae"
server_key_version: "encryptionkey@0"
I20260810 14:53:28.704988 3448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:28.708582 3448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:28.708922 3448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:28.709060 3448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:28.713199 3448 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:42777
I20260810 14:53:28.713236 3518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:42777 every 8 connection(s)
I20260810 14:53:28.713586 3448 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:28.714262 3520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:28.717231 3520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3: Bootstrap starting.
I20260810 14:53:28.717784 3520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:28.718026 3520 log.cc:824] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:28.718719 3520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3: No bootstrap required, opened a new log
I20260810 14:53:28.720459 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 3448
I20260810 14:53:28.720577 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:28.720472 3520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ffe77f95fb74174b1c4cfcb2163a8a3" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42777 } }
I20260810 14:53:28.720681 3520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:28.720729 3520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ffe77f95fb74174b1c4cfcb2163a8a3, State: Initialized, Role: FOLLOWER
I20260810 14:53:28.720791 31017 external_mini_cluster.cc:1428] Setting key add4da6252ac15abe2f788cc44977a23
I20260810 14:53:28.720958 3520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ffe77f95fb74174b1c4cfcb2163a8a3" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42777 } }
I20260810 14:53:28.721047 3520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:28.721094 3520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:28.721144 3520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:28.721927 3520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ffe77f95fb74174b1c4cfcb2163a8a3" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42777 } }
I20260810 14:53:28.722047 3520 leader_election.cc:302] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ffe77f95fb74174b1c4cfcb2163a8a3; no voters:
I20260810 14:53:28.722569 3520 leader_election.cc:288] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:28.722617 3525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:28.722793 3525 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [term 1 LEADER]: Becoming Leader. State: Replica: 7ffe77f95fb74174b1c4cfcb2163a8a3, State: Running, Role: LEADER
I20260810 14:53:28.722879 3520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:28.722899 3525 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ffe77f95fb74174b1c4cfcb2163a8a3" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42777 } }
I20260810 14:53:28.723248 3525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ffe77f95fb74174b1c4cfcb2163a8a3. Latest consensus state: current_term: 1 leader_uuid: "7ffe77f95fb74174b1c4cfcb2163a8a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ffe77f95fb74174b1c4cfcb2163a8a3" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42777 } } }
I20260810 14:53:28.723330 3525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:28.723270 3527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ffe77f95fb74174b1c4cfcb2163a8a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ffe77f95fb74174b1c4cfcb2163a8a3" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42777 } } }
I20260810 14:53:28.723516 3527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:28.723572 3532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:28.723603 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.721331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39306 (local address 127.30.74.126:42777)
0810 14:53:28.721547 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:28.721554 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:28.721581 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:53:28.721679 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:28.721684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:28.721718 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:28.721835 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:28.722559 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:28.722564 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:28.722564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:53:28.722569 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:28.722624 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:53:28.722639 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:28.722713 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:28.722964 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 14:53:28.723024 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260810 14:53:28.724372 3532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:28.726181 3532 catalog_manager.cc:1347] Generated new cluster ID: f7b84d26e14f413984100be412c7dca1
I20260810 14:53:28.726244 3532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:53:28.748427 3532 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:28.748653 3532 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:28.753017 3532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3: Generated new TSK 0
I20260810 14:53:28.753225 3532 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:53:28.763382 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42777
--builtin_ntp_servers=127.30.74.84:33993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 14:53:28.864214 3545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:28.864506 3545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:28.864552 3545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:28.865834 3545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:28.865911 3545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:28.865957 3545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:53:28.867452 3545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:28.867753 3545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:28.868084 3545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:28.874128 3553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:28.874177 3551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:28.874137 3550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:28.874805 3545 server_base.cc:1034] running on GCE node
I20260810 14:53:28.875043 3545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:28.875319 3545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:28.876493 3545 hybrid_clock.cc:648] HybridClock initialized: now 1786373608876475 us; error 34 us; skew 500 ppm
I20260810 14:53:28.877920 3545 webserver.cc:460] Webserver started at http://127.30.74.65:33713/ using document root <none> and password file <none>
I20260810 14:53:28.878189 3545 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:28.878268 3545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:28.878429 3545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:28.879554 3545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "9429e01d2c6444fb925fa8b47d9d71fb"
format_stamp: "Formatted at 2026-08-10 14:53:28 on dist-test-slave-0b58"
server_key: "be6b76a2c9ff2cd63be6c2d2303288b0"
server_key_iv: "4efbd5dca06b71c655b5165ef33f8e61"
server_key_version: "encryptionkey@0"
I20260810 14:53:28.879969 3545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "9429e01d2c6444fb925fa8b47d9d71fb"
format_stamp: "Formatted at 2026-08-10 14:53:28 on dist-test-slave-0b58"
server_key: "be6b76a2c9ff2cd63be6c2d2303288b0"
server_key_iv: "4efbd5dca06b71c655b5165ef33f8e61"
server_key_version: "encryptionkey@0"
I20260810 14:53:28.881415 3545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:53:28.882239 3560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:28.882440 3545 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:28.882552 3545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "9429e01d2c6444fb925fa8b47d9d71fb"
format_stamp: "Formatted at 2026-08-10 14:53:28 on dist-test-slave-0b58"
server_key: "be6b76a2c9ff2cd63be6c2d2303288b0"
server_key_iv: "4efbd5dca06b71c655b5165ef33f8e61"
server_key_version: "encryptionkey@0"
I20260810 14:53:28.882632 3545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:28.886138 3545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:28.886415 3545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:28.886559 3545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:28.886787 3545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:28.887099 3545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:28.887156 3545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:28.887195 3545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:28.887223 3545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:28.894194 3545 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:35083
I20260810 14:53:28.894273 3680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:35083 every 8 connection(s)
I20260810 14:53:28.894603 3545 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:53:28.897926 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.895425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:60155 (local address 127.30.74.126:42777)
0810 14:53:28.895499 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:28.895502 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:28.895786 (+ 284us) server_negotiation.cc:407] Connection header received
0810 14:53:28.896297 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:28.896298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:28.896307 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:28.896380 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:28.897672 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:28.897673 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:28.897673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:53:28.897677 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:28.897693 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:53:28.897697 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:28.897755 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:28.897857 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 14:53:28.897873 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 14:53:28.898106 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:28.895388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42777 (local address 127.30.74.65:60155)
0810 14:53:28.895652 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:28.895701 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:28.896174 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:28.896441 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:28.896444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:28.896453 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:28.897379 (+ 926us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:53:28.897380 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:28.897410 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:53:28.897438 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:28.897776 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:28.897781 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:28.897783 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 14:53:28.897849 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 14:53:28.897901 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":106,"threads_started":1}
I20260810 14:53:28.898695 3681 heartbeater.cc:347] Connected to a master server at 127.30.74.126:42777
I20260810 14:53:28.898824 3681 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:28.898985 3681 heartbeater.cc:511] Master 127.30.74.126:42777 requested a full tablet report, sending...
I20260810 14:53:28.899536 3481 ts_manager.cc:194] Registered new tserver with Master: 9429e01d2c6444fb925fa8b47d9d71fb (127.30.74.65:35083)
I20260810 14:53:28.900359 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 3545
I20260810 14:53:28.900454 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:53:28.900620 31017 external_mini_cluster.cc:1428] Setting key 94415c88e3d506fc11cce8f81a18a29a
I20260810 14:53:28.901479 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42777
--builtin_ntp_servers=127.30.74.84:33993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 14:53:28.985441 3684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:28.985651 3684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:28.985677 3684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:28.986907 3684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:28.986948 3684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:28.986974 3684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:53:28.988324 3684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:28.988520 3684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:28.988770 3684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:28.993714 3689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:28.993713 3690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:28.993754 3692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:28.993906 3684 server_base.cc:1034] running on GCE node
I20260810 14:53:28.994172 3684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:28.994424 3684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:28.995568 3684 hybrid_clock.cc:648] HybridClock initialized: now 1786373608995558 us; error 35 us; skew 500 ppm
I20260810 14:53:28.996941 3684 webserver.cc:460] Webserver started at http://127.30.74.66:44149/ using document root <none> and password file <none>
I20260810 14:53:28.997159 3684 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:28.997207 3684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:28.997318 3684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:28.998405 3684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "932ecf85505441209bb6a92866bee407"
format_stamp: "Formatted at 2026-08-10 14:53:28 on dist-test-slave-0b58"
server_key: "bf8592a574f5665e41bb35517904ece4"
server_key_iv: "02f8c361293b9c309b089c0c32843b22"
server_key_version: "encryptionkey@0"
I20260810 14:53:28.998718 3684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "932ecf85505441209bb6a92866bee407"
format_stamp: "Formatted at 2026-08-10 14:53:28 on dist-test-slave-0b58"
server_key: "bf8592a574f5665e41bb35517904ece4"
server_key_iv: "02f8c361293b9c309b089c0c32843b22"
server_key_version: "encryptionkey@0"
I20260810 14:53:28.999933 3684 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:53:29.000689 3699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:29.000886 3684 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:29.000979 3684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "932ecf85505441209bb6a92866bee407"
format_stamp: "Formatted at 2026-08-10 14:53:28 on dist-test-slave-0b58"
server_key: "bf8592a574f5665e41bb35517904ece4"
server_key_iv: "02f8c361293b9c309b089c0c32843b22"
server_key_version: "encryptionkey@0"
I20260810 14:53:29.001034 3684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:29.004325 3684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:29.004549 3684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:29.004644 3684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:29.004805 3684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:29.005054 3684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:29.005085 3684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:29.005105 3684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:29.005118 3684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:29.011443 3684 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:35381
I20260810 14:53:29.011500 3819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:35381 every 8 connection(s)
I20260810 14:53:29.011783 3684 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:53:29.015085 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.012569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:50251 (local address 127.30.74.126:42777)
0810 14:53:29.012646 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:29.012649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:29.012975 (+ 326us) server_negotiation.cc:407] Connection header received
0810 14:53:29.013452 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:29.013453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:29.013459 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:29.013520 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:29.014831 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:29.014833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:29.014833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:53:29.014838 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:29.014855 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:53:29.014859 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:29.014925 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:29.015010 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 14:53:29.015025 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 14:53:29.015297 3822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.012562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42777 (local address 127.30.74.66:50251)
0810 14:53:29.012844 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:29.012879 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:29.013351 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:29.013601 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:29.013604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:29.013613 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:29.014644 (+ 1031us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:53:29.014644 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:29.014676 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:53:29.014709 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:29.014934 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:29.014939 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:29.014941 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 14:53:29.015002 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 14:53:29.015056 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":125,"threads_started":1}
I20260810 14:53:29.015803 3820 heartbeater.cc:347] Connected to a master server at 127.30.74.126:42777
I20260810 14:53:29.015892 3820 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:29.016016 3820 heartbeater.cc:511] Master 127.30.74.126:42777 requested a full tablet report, sending...
I20260810 14:53:29.016392 3481 ts_manager.cc:194] Registered new tserver with Master: 932ecf85505441209bb6a92866bee407 (127.30.74.66:35381)
I20260810 14:53:29.017107 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 3684
I20260810 14:53:29.017195 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:53:29.017377 31017 external_mini_cluster.cc:1428] Setting key 95afb88f5edf4c746b911f7b532ec6ce
I20260810 14:53:29.018339 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42777
--builtin_ntp_servers=127.30.74.84:33993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 14:53:29.102553 3823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:29.102797 3823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:29.102837 3823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:29.104118 3823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:29.104182 3823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:29.104225 3823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:53:29.105666 3823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:29.105933 3823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:29.106249 3823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:29.111407 3829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:29.111559 3823 server_base.cc:1034] running on GCE node
W20260810 14:53:29.111406 3828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:29.111403 3831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:29.111893 3823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:29.112202 3823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:29.113401 3823 hybrid_clock.cc:648] HybridClock initialized: now 1786373609113381 us; error 36 us; skew 500 ppm
I20260810 14:53:29.114691 3823 webserver.cc:460] Webserver started at http://127.30.74.67:35541/ using document root <none> and password file <none>
I20260810 14:53:29.114964 3823 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:29.115037 3823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:29.115207 3823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:29.116437 3823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "a4eabf1487d24cd3aa538b953b046673"
format_stamp: "Formatted at 2026-08-10 14:53:29 on dist-test-slave-0b58"
server_key: "a239d4ac756c7b9c3bd987c3a10c50f4"
server_key_iv: "66c3000f85cfd55ce563a0c4617c1531"
server_key_version: "encryptionkey@0"
I20260810 14:53:29.116818 3823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "a4eabf1487d24cd3aa538b953b046673"
format_stamp: "Formatted at 2026-08-10 14:53:29 on dist-test-slave-0b58"
server_key: "a239d4ac756c7b9c3bd987c3a10c50f4"
server_key_iv: "66c3000f85cfd55ce563a0c4617c1531"
server_key_version: "encryptionkey@0"
I20260810 14:53:29.118330 3823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:53:29.119274 3837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:29.119616 3823 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:29.119750 3823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "a4eabf1487d24cd3aa538b953b046673"
format_stamp: "Formatted at 2026-08-10 14:53:29 on dist-test-slave-0b58"
server_key: "a239d4ac756c7b9c3bd987c3a10c50f4"
server_key_iv: "66c3000f85cfd55ce563a0c4617c1531"
server_key_version: "encryptionkey@0"
I20260810 14:53:29.119851 3823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:29.123917 3823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:29.124222 3823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:29.124413 3823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:29.124657 3823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:29.125010 3823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:29.125069 3823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:29.125113 3823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:29.125149 3823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:29.131356 3823 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:36487
I20260810 14:53:29.131402 3961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:36487 every 8 connection(s)
I20260810 14:53:29.131774 3823 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:53:29.134084 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 3823
I20260810 14:53:29.134217 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:53:29.134459 31017 external_mini_cluster.cc:1428] Setting key 8813fe865f4651b611f3ade98b267ade
I20260810 14:53:29.135413 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.132653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:35729 (local address 127.30.74.126:42777)
0810 14:53:29.132727 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:29.132729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:29.132985 (+ 256us) server_negotiation.cc:407] Connection header received
0810 14:53:29.133494 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:29.133495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:29.133501 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:29.133568 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:29.135098 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:29.135100 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:29.135100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:53:29.135105 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:29.135124 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:53:29.135129 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:29.135190 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:29.135323 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 14:53:29.135340 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 14:53:29.135651 3964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.132637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42777 (local address 127.30.74.67:35729)
0810 14:53:29.132842 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:29.132889 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:29.133397 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:29.133626 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:29.133629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:29.133637 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:29.134867 (+ 1230us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:53:29.134868 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:29.134935 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:53:29.134977 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:29.135215 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:29.135222 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:29.135226 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:29.135319 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 14:53:29.135375 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
I20260810 14:53:29.136210 3962 heartbeater.cc:347] Connected to a master server at 127.30.74.126:42777
I20260810 14:53:29.136343 3962 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:29.136503 3962 heartbeater.cc:511] Master 127.30.74.126:42777 requested a full tablet report, sending...
I20260810 14:53:29.136873 3481 ts_manager.cc:194] Registered new tserver with Master: a4eabf1487d24cd3aa538b953b046673 (127.30.74.67:36487)
I20260810 14:53:29.146446 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:53:29.148981 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.147955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39318 (local address 127.30.74.126:42777)
0810 14:53:29.148021 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:29.148024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:29.148136 (+ 112us) server_negotiation.cc:407] Connection header received
0810 14:53:29.148178 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:29.148179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:29.148183 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:29.148233 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:29.148810 (+ 577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:29.148812 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:29.148812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:53:29.148815 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:29.148831 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:53:29.148834 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:29.148884 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:29.148918 (+ 34us) server_negotiation.cc:299] Negotiation successful
0810 14:53:29.148931 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 14:53:29.150607 3481 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:53:29.152077 3481 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:29.156183 3974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.154334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:36487 (local address 127.0.0.1:36694)
0810 14:53:29.154778 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:29.154788 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:29.154829 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:29.155170 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:29.155177 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:29.155184 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:29.155738 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:53:29.155739 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:29.155757 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:53:29.155784 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:29.156024 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:29.156031 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:29.156034 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:53:29.156097 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 14:53:29.156112 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":102,"threads_started":1}
I20260810 14:53:29.156198 3973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.154443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36694 (local address 127.30.74.67:36487)
0810 14:53:29.154625 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:29.154630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:29.154901 (+ 271us) server_negotiation.cc:407] Connection header received
0810 14:53:29.154933 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:29.154937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:29.154948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:29.155019 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:29.155875 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:29.155885 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:29.155885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:53:29.155890 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:29.155927 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:53:29.155957 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:29.156029 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:29.156102 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 14:53:29.156119 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":75,"threads_started":1}
I20260810 14:53:29.156364 3970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.154325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:35381 (local address 127.0.0.1:42822)
0810 14:53:29.154649 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:29.154706 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:29.154783 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:29.155301 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:29.155301 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:29.155304 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:29.155885 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:53:29.155886 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:29.155894 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:53:29.155907 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:29.156186 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:29.156187 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:29.156187 (+ 0us) client_negotiation.cc:769] Sending connection context
0810 14:53:29.156235 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 14:53:29.156250 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":70,"threads_started":1}
I20260810 14:53:29.156630 3972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.154423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42822 (local address 127.30.74.66:35381)
0810 14:53:29.154625 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:29.154630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:29.154798 (+ 168us) server_negotiation.cc:407] Connection header received
0810 14:53:29.154839 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:29.154841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:29.154856 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:29.154939 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:29.156016 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:29.156024 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:29.156024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:53:29.156029 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:29.156067 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:53:29.156090 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:29.156180 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:29.156504 (+ 324us) server_negotiation.cc:299] Negotiation successful
0810 14:53:29.156522 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260810 14:53:29.157073 3889 tablet_service.cc:1467] Processing CreateTablet for tablet 2e1b51424fdc491f828c04b6d73722a4 (DEFAULT_TABLE table=test-table [id=453c5a2f584e41d388f508c6f03a17eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:29.157373 3971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.154536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:35083 (local address 127.0.0.1:43896)
0810 14:53:29.154875 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:29.154884 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:29.154918 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:29.155341 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:29.155342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:29.155344 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:29.156230 (+ 886us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:53:29.156230 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:29.156667 (+ 437us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:53:29.156682 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:29.157203 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:29.157204 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:29.157205 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 14:53:29.157253 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 14:53:29.157281 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":82,"threads_started":1}
I20260810 14:53:29.157291 3743 tablet_service.cc:1467] Processing CreateTablet for tablet 2e1b51424fdc491f828c04b6d73722a4 (DEFAULT_TABLE table=test-table [id=453c5a2f584e41d388f508c6f03a17eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:29.157518 3889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e1b51424fdc491f828c04b6d73722a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:29.157626 3976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.154791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43896 (local address 127.30.74.65:35083)
0810 14:53:29.155004 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:29.155008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:29.155018 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:29.155044 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:29.155046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:29.155054 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:29.155127 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:29.157035 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:29.157042 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:29.157043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:53:29.157048 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:29.157084 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:53:29.157109 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:29.157174 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:29.157529 (+ 355us) server_negotiation.cc:299] Negotiation successful
0810 14:53:29.157548 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":84,"threads_started":1}
I20260810 14:53:29.157666 3743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e1b51424fdc491f828c04b6d73722a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:29.158217 3612 tablet_service.cc:1467] Processing CreateTablet for tablet 2e1b51424fdc491f828c04b6d73722a4 (DEFAULT_TABLE table=test-table [id=453c5a2f584e41d388f508c6f03a17eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:29.158530 3612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e1b51424fdc491f828c04b6d73722a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:29.160223 3977 tablet_bootstrap.cc:492] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673: Bootstrap starting.
I20260810 14:53:29.160862 3977 tablet_bootstrap.cc:654] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:29.161167 3978 tablet_bootstrap.cc:492] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407: Bootstrap starting.
I20260810 14:53:29.161164 3977 log.cc:824] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:29.161808 3978 tablet_bootstrap.cc:654] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:29.161931 3980 tablet_bootstrap.cc:492] T 2e1b51424fdc491f828c04b6d73722a4 P 9429e01d2c6444fb925fa8b47d9d71fb: Bootstrap starting.
I20260810 14:53:29.162128 3978 log.cc:824] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:29.162364 3977 tablet_bootstrap.cc:492] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673: No bootstrap required, opened a new log
I20260810 14:53:29.162442 3977 ts_tablet_manager.cc:1397] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:53:29.162649 3980 tablet_bootstrap.cc:654] T 2e1b51424fdc491f828c04b6d73722a4 P 9429e01d2c6444fb925fa8b47d9d71fb: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:29.162969 3978 tablet_bootstrap.cc:492] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407: No bootstrap required, opened a new log
I20260810 14:53:29.162986 3980 log.cc:824] T 2e1b51424fdc491f828c04b6d73722a4 P 9429e01d2c6444fb925fa8b47d9d71fb: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:29.163046 3978 ts_tablet_manager.cc:1397] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:53:29.163805 3980 tablet_bootstrap.cc:492] T 2e1b51424fdc491f828c04b6d73722a4 P 9429e01d2c6444fb925fa8b47d9d71fb: No bootstrap required, opened a new log
I20260810 14:53:29.163880 3980 ts_tablet_manager.cc:1397] T 2e1b51424fdc491f828c04b6d73722a4 P 9429e01d2c6444fb925fa8b47d9d71fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:53:29.164914 3977 raft_consensus.cc:348] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932ecf85505441209bb6a92866bee407" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35381 } } peers { permanent_uuid: "9429e01d2c6444fb925fa8b47d9d71fb" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35083 } } peers { permanent_uuid: "a4eabf1487d24cd3aa538b953b046673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36487 } }
I20260810 14:53:29.165177 3977 raft_consensus.cc:374] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:29.165228 3977 raft_consensus.cc:729] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4eabf1487d24cd3aa538b953b046673, State: Initialized, Role: FOLLOWER
I20260810 14:53:29.165383 3977 consensus_queue.cc:260] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932ecf85505441209bb6a92866bee407" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35381 } } peers { permanent_uuid: "9429e01d2c6444fb925fa8b47d9d71fb" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35083 } } peers { permanent_uuid: "a4eabf1487d24cd3aa538b953b046673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36487 } }
I20260810 14:53:29.165671 3977 ts_tablet_manager.cc:1428] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:53:29.165822 3962 heartbeater.cc:503] Master 127.30.74.126:42777 was elected leader, sending a full tablet report...
I20260810 14:53:29.165912 3978 raft_consensus.cc:348] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932ecf85505441209bb6a92866bee407" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35381 } } peers { permanent_uuid: "9429e01d2c6444fb925fa8b47d9d71fb" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35083 } } peers { permanent_uuid: "a4eabf1487d24cd3aa538b953b046673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36487 } }
I20260810 14:53:29.166121 3978 raft_consensus.cc:374] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:29.166178 3978 raft_consensus.cc:729] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 932ecf85505441209bb6a92866bee407, State: Initialized, Role: FOLLOWER
I20260810 14:53:29.166317 3978 consensus_queue.cc:260] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932ecf85505441209bb6a92866bee407" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35381 } } peers { permanent_uuid: "9429e01d2c6444fb925fa8b47d9d71fb" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35083 } } peers { permanent_uuid: "a4eabf1487d24cd3aa538b953b046673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36487 } }
I20260810 14:53:29.166622 3978 ts_tablet_manager.cc:1428] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 14:53:29.166647 3820 heartbeater.cc:503] Master 127.30.74.126:42777 was elected leader, sending a full tablet report...
I20260810 14:53:29.166989 3980 raft_consensus.cc:348] T 2e1b51424fdc491f828c04b6d73722a4 P 9429e01d2c6444fb925fa8b47d9d71fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932ecf85505441209bb6a92866bee407" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35381 } } peers { permanent_uuid: "9429e01d2c6444fb925fa8b47d9d71fb" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35083 } } peers { permanent_uuid: "a4eabf1487d24cd3aa538b953b046673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36487 } }
I20260810 14:53:29.167117 3980 raft_consensus.cc:374] T 2e1b51424fdc491f828c04b6d73722a4 P 9429e01d2c6444fb925fa8b47d9d71fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:29.167166 3980 raft_consensus.cc:729] T 2e1b51424fdc491f828c04b6d73722a4 P 9429e01d2c6444fb925fa8b47d9d71fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9429e01d2c6444fb925fa8b47d9d71fb, State: Initialized, Role: FOLLOWER
I20260810 14:53:29.167277 3980 consensus_queue.cc:260] T 2e1b51424fdc491f828c04b6d73722a4 P 9429e01d2c6444fb925fa8b47d9d71fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932ecf85505441209bb6a92866bee407" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35381 } } peers { permanent_uuid: "9429e01d2c6444fb925fa8b47d9d71fb" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35083 } } peers { permanent_uuid: "a4eabf1487d24cd3aa538b953b046673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36487 } }
I20260810 14:53:29.167562 3980 ts_tablet_manager.cc:1428] T 2e1b51424fdc491f828c04b6d73722a4 P 9429e01d2c6444fb925fa8b47d9d71fb: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 14:53:29.167584 3681 heartbeater.cc:503] Master 127.30.74.126:42777 was elected leader, sending a full tablet report...
W20260810 14:53:29.262319 3821 tablet.cc:2369] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:53:29.382403 3963 tablet.cc:2369] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:53:29.388190 3983 raft_consensus.cc:482] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:29.388325 3983 raft_consensus.cc:504] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932ecf85505441209bb6a92866bee407" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35381 } } peers { permanent_uuid: "9429e01d2c6444fb925fa8b47d9d71fb" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35083 } } peers { permanent_uuid: "a4eabf1487d24cd3aa538b953b046673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36487 } }
I20260810 14:53:29.388727 3983 leader_election.cc:288] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 932ecf85505441209bb6a92866bee407 (127.30.74.66:35381), 9429e01d2c6444fb925fa8b47d9d71fb (127.30.74.65:35083)
I20260810 14:53:29.390064 3964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.388922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:35381 (local address 127.30.74.67:51663)
0810 14:53:29.388987 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:29.389000 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:29.389084 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:29.389253 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:29.389255 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:29.389258 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:29.389702 (+ 444us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:53:29.389703 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:29.389710 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:53:29.389723 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:29.389931 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:29.389932 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:29.389934 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 14:53:29.389989 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 14:53:29.390001 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 14:53:29.390103 3972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.389043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:51663 (local address 127.30.74.66:35381)
0810 14:53:29.389099 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:29.389102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:29.389113 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:53:29.389146 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:29.389147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:29.389152 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:29.389213 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:29.389832 (+ 619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:29.389833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:29.389834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:53:29.389838 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:29.389864 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:53:29.389868 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:29.389916 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:29.389995 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 14:53:29.390011 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 14:53:29.390259 3976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.389152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:55571 (local address 127.30.74.65:35083)
0810 14:53:29.389200 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:29.389202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:29.389279 (+ 77us) server_negotiation.cc:407] Connection header received
0810 14:53:29.389310 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:29.389311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:29.389315 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:29.389358 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:29.389992 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:29.389994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:29.389994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:53:29.389999 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:29.390015 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:53:29.390020 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:29.390076 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:29.390177 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 14:53:29.390193 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:53:29.390410 3770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e1b51424fdc491f828c04b6d73722a4" candidate_uuid: "a4eabf1487d24cd3aa538b953b046673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932ecf85505441209bb6a92866bee407" is_pre_election: true
I20260810 14:53:29.390545 3987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.389040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:35083 (local address 127.30.74.67:55571)
0810 14:53:29.389198 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:29.389208 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:29.389254 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:29.389382 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:29.389383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:29.389385 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:29.389854 (+ 469us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:53:29.389855 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:29.389860 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:53:29.389869 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:29.390083 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:29.390084 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:29.390085 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 14:53:29.390464 (+ 379us) client_negotiation.cc:240] Negotiation successful
0810 14:53:29.390479 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":98,"threads_started":1}
I20260810 14:53:29.390568 3770 raft_consensus.cc:2454] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4eabf1487d24cd3aa538b953b046673 in term 0.
I20260810 14:53:29.390817 3839 leader_election.cc:302] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 932ecf85505441209bb6a92866bee407, a4eabf1487d24cd3aa538b953b046673; no voters:
I20260810 14:53:29.390960 3983 raft_consensus.cc:2781] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:29.390944 3633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e1b51424fdc491f828c04b6d73722a4" candidate_uuid: "a4eabf1487d24cd3aa538b953b046673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9429e01d2c6444fb925fa8b47d9d71fb" is_pre_election: true
I20260810 14:53:29.391031 3983 raft_consensus.cc:482] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:29.391068 3983 raft_consensus.cc:3037] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:29.391096 3633 raft_consensus.cc:2454] T 2e1b51424fdc491f828c04b6d73722a4 P 9429e01d2c6444fb925fa8b47d9d71fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4eabf1487d24cd3aa538b953b046673 in term 0.
I20260810 14:53:29.391817 3983 raft_consensus.cc:504] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932ecf85505441209bb6a92866bee407" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35381 } } peers { permanent_uuid: "9429e01d2c6444fb925fa8b47d9d71fb" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35083 } } peers { permanent_uuid: "a4eabf1487d24cd3aa538b953b046673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36487 } }
I20260810 14:53:29.391959 3983 leader_election.cc:288] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [CANDIDATE]: Term 1 election: Requested vote from peers 932ecf85505441209bb6a92866bee407 (127.30.74.66:35381), 9429e01d2c6444fb925fa8b47d9d71fb (127.30.74.65:35083)
I20260810 14:53:29.392133 3633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e1b51424fdc491f828c04b6d73722a4" candidate_uuid: "a4eabf1487d24cd3aa538b953b046673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9429e01d2c6444fb925fa8b47d9d71fb"
I20260810 14:53:29.392135 3770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e1b51424fdc491f828c04b6d73722a4" candidate_uuid: "a4eabf1487d24cd3aa538b953b046673" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "932ecf85505441209bb6a92866bee407"
I20260810 14:53:29.392201 3633 raft_consensus.cc:3037] T 2e1b51424fdc491f828c04b6d73722a4 P 9429e01d2c6444fb925fa8b47d9d71fb [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:29.392247 3770 raft_consensus.cc:3037] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:29.392948 3633 raft_consensus.cc:2454] T 2e1b51424fdc491f828c04b6d73722a4 P 9429e01d2c6444fb925fa8b47d9d71fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4eabf1487d24cd3aa538b953b046673 in term 1.
I20260810 14:53:29.393069 3770 raft_consensus.cc:2454] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4eabf1487d24cd3aa538b953b046673 in term 1.
I20260810 14:53:29.393111 3839 leader_election.cc:302] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9429e01d2c6444fb925fa8b47d9d71fb, a4eabf1487d24cd3aa538b953b046673; no voters:
I20260810 14:53:29.393224 3983 raft_consensus.cc:2781] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:29.393383 3983 raft_consensus.cc:686] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [term 1 LEADER]: Becoming Leader. State: Replica: a4eabf1487d24cd3aa538b953b046673, State: Running, Role: LEADER
I20260810 14:53:29.393483 3983 consensus_queue.cc:237] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932ecf85505441209bb6a92866bee407" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35381 } } peers { permanent_uuid: "9429e01d2c6444fb925fa8b47d9d71fb" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35083 } } peers { permanent_uuid: "a4eabf1487d24cd3aa538b953b046673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36487 } }
I20260810 14:53:29.394272 3481 catalog_manager.cc:5495] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4eabf1487d24cd3aa538b953b046673 (127.30.74.67). New cstate: current_term: 1 leader_uuid: "a4eabf1487d24cd3aa538b953b046673" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "932ecf85505441209bb6a92866bee407" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9429e01d2c6444fb925fa8b47d9d71fb" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4eabf1487d24cd3aa538b953b046673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36487 } health_report { overall_health: HEALTHY } } }
W20260810 14:53:29.395300 3682 tablet.cc:2369] T 2e1b51424fdc491f828c04b6d73722a4 P 9429e01d2c6444fb925fa8b47d9d71fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:53:29.442217 3973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.441090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36708 (local address 127.30.74.67:36487)
0810 14:53:29.441173 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:29.441175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:29.441184 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:29.441236 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:29.441237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:29.441253 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:29.441319 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:29.441924 (+ 605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:29.441930 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:29.441931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:53:29.441936 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:29.441956 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:53:29.441961 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:29.442031 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:29.442098 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 14:53:29.442113 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 14:53:29.444236 3770 raft_consensus.cc:1261] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407 [term 1 FOLLOWER]: Refusing update from remote peer a4eabf1487d24cd3aa538b953b046673: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:29.444319 3633 raft_consensus.cc:1261] T 2e1b51424fdc491f828c04b6d73722a4 P 9429e01d2c6444fb925fa8b47d9d71fb [term 1 FOLLOWER]: Refusing update from remote peer a4eabf1487d24cd3aa538b953b046673: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:29.444644 3988 consensus_queue.cc:1035] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "932ecf85505441209bb6a92866bee407" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:29.444792 3983 consensus_queue.cc:1035] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9429e01d2c6444fb925fa8b47d9d71fb" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:29.447724 3992 mvcc.cc:205] Tried to move back new op lower bound from 7316986304280559616 to 7316986304076308480. Current Snapshot: MvccSnapshot[applied={T|T < 7316986304280559616}]
I20260810 14:53:29.448279 3993 mvcc.cc:205] Tried to move back new op lower bound from 7316986304280559616 to 7316986304076308480. Current Snapshot: MvccSnapshot[applied={T|T < 7316986304280559616 or (T in {7316986304280559616})}]
I20260810 14:53:29.449643 3479 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:29.449745 3479 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:29.450572 3479 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ffe77f95fb74174b1c4cfcb2163a8a3: Sending DeleteTablet for 3 replicas of tablet 2e1b51424fdc491f828c04b6d73722a4
I20260810 14:53:29.450876 3743 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e1b51424fdc491f828c04b6d73722a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:29 UTC) from {username='slave'} at 127.0.0.1:42822
I20260810 14:53:29.450953 3889 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e1b51424fdc491f828c04b6d73722a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:29 UTC) from {username='slave'} at 127.0.0.1:36694
I20260810 14:53:29.450973 3612 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e1b51424fdc491f828c04b6d73722a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:29 UTC) from {username='slave'} at 127.0.0.1:43896
I20260810 14:53:29.451197 4002 tablet_replica.cc:330] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673: stopping tablet replica
I20260810 14:53:29.451303 4002 raft_consensus.cc:2229] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:53:29.451426 4002 raft_consensus.cc:2258] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:29.451480 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 3545
I20260810 14:53:29.451437 4001 tablet_replica.cc:330] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407: stopping tablet replica
I20260810 14:53:29.452389 4002 ts_tablet_manager.cc:1905] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:29.453471 4001 raft_consensus.cc:2229] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:29.453632 4001 raft_consensus.cc:2258] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:29.454118 4002 ts_tablet_manager.cc:1918] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:29.454196 4002 log.cc:1196] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/2e1b51424fdc491f828c04b6d73722a4
I20260810 14:53:29.454478 4002 ts_tablet_manager.cc:1939] T 2e1b51424fdc491f828c04b6d73722a4 P a4eabf1487d24cd3aa538b953b046673: Deleting consensus metadata
I20260810 14:53:29.454604 4001 ts_tablet_manager.cc:1905] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:29.454898 3464 catalog_manager.cc:4841] TS a4eabf1487d24cd3aa538b953b046673 (127.30.74.67:36487): tablet 2e1b51424fdc491f828c04b6d73722a4 (table test-table [id=453c5a2f584e41d388f508c6f03a17eb]) successfully deleted
I20260810 14:53:29.456776 4001 ts_tablet_manager.cc:1918] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:29.456858 4001 log.cc:1196] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/2e1b51424fdc491f828c04b6d73722a4
I20260810 14:53:29.457103 4001 ts_tablet_manager.cc:1939] T 2e1b51424fdc491f828c04b6d73722a4 P 932ecf85505441209bb6a92866bee407: Deleting consensus metadata
I20260810 14:53:29.457742 3465 catalog_manager.cc:4841] TS 932ecf85505441209bb6a92866bee407 (127.30.74.66:35381): tablet 2e1b51424fdc491f828c04b6d73722a4 (table test-table [id=453c5a2f584e41d388f508c6f03a17eb]) successfully deleted
W20260810 14:53:29.460023 3465 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:35083 (error 108)
I20260810 14:53:29.460323 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 3684
I20260810 14:53:29.467576 3971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.467404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:35083 (local address 127.0.0.1:43910)
0810 14:53:29.467480 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:29.467536 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:35083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 14:53:29.467796 3465 catalog_manager.cc:4584] TS 9429e01d2c6444fb925fa8b47d9d71fb (127.30.74.65:35083): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e1b51424fdc491f828c04b6d73722a4: Network error: Client connection negotiation failed: client connection to 127.30.74.65:35083: connect: Connection refused (error 111)
I20260810 14:53:29.472057 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 3823
I20260810 14:53:29.478992 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 3448
2026-08-10T14:53:29Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (903 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 14:53:29.500847 31017 mini_oidc.cc:152] Starting JWKS server
I20260810 14:53:29.501966 31017 webserver.cc:460] Webserver started at https://127.0.0.1:41203/ using document root <none> and password file <none>
I20260810 14:53:29.502075 31017 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 14:53:29.502261 31017 webserver.cc:460] Webserver started at http://127.0.0.1:43053/ using document root <none> and password file <none>
2026-08-10T14:53:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:29Z Disabled control of system clock
I20260810 14:53:29.511224 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:43355
--webserver_interface=127.30.74.126
--webserver_port=0
--builtin_ntp_servers=127.30.74.84:38371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:43355
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41203/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/testchainca.cert with env {}
W20260810 14:53:29.593667 4010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:29.593880 4010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:29.593902 4010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:29.595023 4010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:29.595062 4010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:29.595074 4010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:29.595088 4010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:29.595099 4010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 14:53:29.595111 4010 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 14:53:29.595124 4010 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41203/jwks.json
I20260810 14:53:29.596817 4010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:38371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:43355
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:43355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41203/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:29.597114 4010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:29.597304 4010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:29.602286 4016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:29.602270 4018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:29.602272 4015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:29.602553 4010 server_base.cc:1034] running on GCE node
I20260810 14:53:29.602958 4010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:29.603268 4010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:29.604423 4010 hybrid_clock.cc:648] HybridClock initialized: now 1786373609604394 us; error 38 us; skew 500 ppm
I20260810 14:53:29.605774 4010 webserver.cc:460] Webserver started at http://127.30.74.126:38561/ using document root <none> and password file <none>
I20260810 14:53:29.606060 4010 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:29.606145 4010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:29.606303 4010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:29.607370 4010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "0a1e8af8afa74ef6bbb7b7b4d6888e98"
format_stamp: "Formatted at 2026-08-10 14:53:29 on dist-test-slave-0b58"
server_key: "d82e60af8948e1e975a5180401110678"
server_key_iv: "db25e7c2b67e97266f3dfd4426c7df67"
server_key_version: "encryptionkey@0"
I20260810 14:53:29.607744 4010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "0a1e8af8afa74ef6bbb7b7b4d6888e98"
format_stamp: "Formatted at 2026-08-10 14:53:29 on dist-test-slave-0b58"
server_key: "d82e60af8948e1e975a5180401110678"
server_key_iv: "db25e7c2b67e97266f3dfd4426c7df67"
server_key_version: "encryptionkey@0"
I20260810 14:53:29.609211 4010 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:53:29.609938 4024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:29.610103 4010 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:29.610213 4010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "0a1e8af8afa74ef6bbb7b7b4d6888e98"
format_stamp: "Formatted at 2026-08-10 14:53:29 on dist-test-slave-0b58"
server_key: "d82e60af8948e1e975a5180401110678"
server_key_iv: "db25e7c2b67e97266f3dfd4426c7df67"
server_key_version: "encryptionkey@0"
I20260810 14:53:29.610291 4010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:29.625869 4010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:29.626187 4010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:29.626309 4010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:29.630321 4010 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:43355
I20260810 14:53:29.630358 4081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:43355 every 8 connection(s)
I20260810 14:53:29.630725 4010 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:29.631753 4082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:29.634205 4082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98: Bootstrap starting.
I20260810 14:53:29.634764 4082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:29.635011 4082 log.cc:824] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:29.635711 4082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98: No bootstrap required, opened a new log
I20260810 14:53:29.636052 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 4010
I20260810 14:53:29.636180 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:29.636417 31017 external_mini_cluster.cc:1428] Setting key f2044a85a362cbc35f8f322e2b3b2c52
I20260810 14:53:29.638259 4082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1e8af8afa74ef6bbb7b7b4d6888e98" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43355 } }
I20260810 14:53:29.638438 4082 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:29.638478 4082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1e8af8afa74ef6bbb7b7b4d6888e98, State: Initialized, Role: FOLLOWER
I20260810 14:53:29.638599 4082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1e8af8afa74ef6bbb7b7b4d6888e98" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43355 } }
I20260810 14:53:29.638679 4082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:29.638720 4082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:29.638764 4082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:29.639521 4082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1e8af8afa74ef6bbb7b7b4d6888e98" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43355 } }
I20260810 14:53:29.639642 4082 leader_election.cc:302] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a1e8af8afa74ef6bbb7b7b4d6888e98; no voters:
I20260810 14:53:29.639812 4082 leader_election.cc:288] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:29.640005 4088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:29.640086 4082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:29.640857 4088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [term 1 LEADER]: Becoming Leader. State: Replica: 0a1e8af8afa74ef6bbb7b7b4d6888e98, State: Running, Role: LEADER
I20260810 14:53:29.640983 4088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1e8af8afa74ef6bbb7b7b4d6888e98" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43355 } }
I20260810 14:53:29.641378 4092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a1e8af8afa74ef6bbb7b7b4d6888e98. Latest consensus state: current_term: 1 leader_uuid: "0a1e8af8afa74ef6bbb7b7b4d6888e98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1e8af8afa74ef6bbb7b7b4d6888e98" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43355 } } }
I20260810 14:53:29.641472 4092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:29.641548 4087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a1e8af8afa74ef6bbb7b7b4d6888e98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a1e8af8afa74ef6bbb7b7b4d6888e98" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43355 } } }
I20260810 14:53:29.641645 4087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98 [sys.catalog]: This master's current role is: LEADER
W20260810 14:53:29.642180 4103 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:53:29.642237 4103 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:53:29.642290 4102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:29.642501 4102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:29.644063 4086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.636958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53284 (local address 127.30.74.126:43355)
0810 14:53:29.637214 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:29.637225 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:29.637259 (+ 34us) server_negotiation.cc:407] Connection header received
0810 14:53:29.637814 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:29.637824 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:29.637905 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:29.638061 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:29.638381 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:29.640225 (+ 1844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:29.641318 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:29.642379 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:29.642504 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:29.643284 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:29.643288 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:29.643289 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:53:29.643522 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:29.643566 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:53:29.643581 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:29.643651 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:29.643755 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 14:53:29.643845 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":100,"threads_started":1}
I20260810 14:53:29.644510 4102 catalog_manager.cc:1347] Generated new cluster ID: a54318ed323f4cabbee497aee5ecac61
I20260810 14:53:29.644564 4102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:53:29.648777 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:53:29.661885 4102 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:29.662460 4102 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:29.669256 4102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a1e8af8afa74ef6bbb7b7b4d6888e98: Generated new TSK 0
I20260810 14:53:29.669468 4102 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 14:53:29.695626 4115 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 14:53:29.695787 4115 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 14:53:29.696031 4086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.680219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53296 (local address 127.30.74.126:43355)
0810 14:53:29.680293 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:29.680296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:29.680460 (+ 164us) server_negotiation.cc:407] Connection header received
0810 14:53:29.680570 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:29.680571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:29.680581 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:29.680642 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0810 14:53:29.680870 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:29.681388 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:29.682566 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:29.682778 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:29.682833 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:29.683110 (+ 277us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 14:53:29.695787 (+ 12677us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41203/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 14:53:29.695979 (+ 192us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53296: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41203/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 14:53:29.696105 4086 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53296: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41203/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 14:53:29.696712 4114 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:29.680117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43355 (local address 127.0.0.1:53296)
0810 14:53:29.680378 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:29.680392 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:29.680472 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:29.680690 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:29.680691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:29.680695 (+ 4us) client_negotiation.cc:189] Negotiated authn=JWT
0810 14:53:29.680785 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:29.680789 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:29.681481 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:29.681482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:29.682472 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:29.682476 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:29.682882 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:29.682883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:29.682956 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:29.682986 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 14:53:29.696013 (+ 13027us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41203/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 14:53:29.696277 (+ 264us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.74.126:43355: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41203/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":115,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.74.126:43355: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41203/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 14:53:29.697448 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 4010
2026-08-10T14:53:29Z chronyd exiting
I20260810 14:53:29.736948 31017 test_util.cc:175] -----------------------------------------------
I20260810 14:53:29.737035 31017 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373586526129-31017-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (237 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 14:53:29.737907 31017 mini_oidc.cc:152] Starting JWKS server
I20260810 14:53:29.738612 31017 webserver.cc:460] Webserver started at https://127.0.0.1:43239/ using document root <none> and password file <none>
I20260810 14:53:29.738654 31017 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 14:53:29.738844 31017 webserver.cc:460] Webserver started at http://127.0.0.1:44209/ using document root <none> and password file <none>
2026-08-10T14:53:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:29Z Disabled control of system clock
I20260810 14:53:29.747502 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:42259
--webserver_interface=127.30.74.126
--webserver_port=0
--builtin_ntp_servers=127.30.74.84:37913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:42259
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43239/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/testchainca.cert with env {}
W20260810 14:53:29.829679 4121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:29.829960 4121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:29.830000 4121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:29.831233 4121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:29.831321 4121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:29.831343 4121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:29.831367 4121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:29.831394 4121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 14:53:29.831426 4121 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 14:53:29.831453 4121 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43239/jwks.json
I20260810 14:53:29.833123 4121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:42259
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:42259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43239/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:29.833432 4121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:29.833683 4121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:29.838800 4130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:29.838809 4128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:29.838862 4127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:29.838976 4121 server_base.cc:1034] running on GCE node
I20260810 14:53:29.839349 4121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:29.839746 4121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:29.840950 4121 hybrid_clock.cc:648] HybridClock initialized: now 1786373609840918 us; error 46 us; skew 500 ppm
I20260810 14:53:29.842365 4121 webserver.cc:460] Webserver started at http://127.30.74.126:43957/ using document root <none> and password file <none>
I20260810 14:53:29.842638 4121 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:29.842708 4121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:29.842875 4121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:29.843995 4121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "0a9223a43fab407b8fbe9e5a9536eb99"
format_stamp: "Formatted at 2026-08-10 14:53:29 on dist-test-slave-0b58"
server_key: "8c93963cb633fa188bdf947c352956a2"
server_key_iv: "7010a704335f7fdf2982917c14f08f8c"
server_key_version: "encryptionkey@0"
I20260810 14:53:29.844442 4121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "0a9223a43fab407b8fbe9e5a9536eb99"
format_stamp: "Formatted at 2026-08-10 14:53:29 on dist-test-slave-0b58"
server_key: "8c93963cb633fa188bdf947c352956a2"
server_key_iv: "7010a704335f7fdf2982917c14f08f8c"
server_key_version: "encryptionkey@0"
I20260810 14:53:29.845839 4121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:29.846647 4136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:29.846827 4121 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 14:53:29.846935 4121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "0a9223a43fab407b8fbe9e5a9536eb99"
format_stamp: "Formatted at 2026-08-10 14:53:29 on dist-test-slave-0b58"
server_key: "8c93963cb633fa188bdf947c352956a2"
server_key_iv: "7010a704335f7fdf2982917c14f08f8c"
server_key_version: "encryptionkey@0"
I20260810 14:53:29.847023 4121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:29.862973 4121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:29.863351 4121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:29.863520 4121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:29.867653 4121 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:42259
I20260810 14:53:29.867692 4191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:42259 every 8 connection(s)
I20260810 14:53:29.868052 4121 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:29.868816 4192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:29.871603 4192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99: Bootstrap starting.
I20260810 14:53:29.871965 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 4121
I20260810 14:53:29.872066 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:29.872244 4192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:29.872303 31017 external_mini_cluster.cc:1428] Setting key a6b9bc169c19d032a1f5be561f037c88
I20260810 14:53:29.872618 4192 log.cc:824] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:29.873385 4192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99: No bootstrap required, opened a new log
I20260810 14:53:29.875650 4192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a9223a43fab407b8fbe9e5a9536eb99" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42259 } }
I20260810 14:53:29.875849 4192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:29.875886 4192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a9223a43fab407b8fbe9e5a9536eb99, State: Initialized, Role: FOLLOWER
I20260810 14:53:29.876017 4192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a9223a43fab407b8fbe9e5a9536eb99" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42259 } }
I20260810 14:53:29.876125 4192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:29.876170 4192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:29.876223 4192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:29.876973 4192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a9223a43fab407b8fbe9e5a9536eb99" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42259 } }
I20260810 14:53:29.877094 4192 leader_election.cc:302] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a9223a43fab407b8fbe9e5a9536eb99; no voters:
I20260810 14:53:29.877310 4192 leader_election.cc:288] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:29.877341 4197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:29.877586 4192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:29.877610 4197 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [term 1 LEADER]: Becoming Leader. State: Replica: 0a9223a43fab407b8fbe9e5a9536eb99, State: Running, Role: LEADER
I20260810 14:53:29.877785 4197 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a9223a43fab407b8fbe9e5a9536eb99" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42259 } }
I20260810 14:53:29.878163 4198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a9223a43fab407b8fbe9e5a9536eb99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a9223a43fab407b8fbe9e5a9536eb99" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42259 } } }
I20260810 14:53:29.878183 4199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a9223a43fab407b8fbe9e5a9536eb99. Latest consensus state: current_term: 1 leader_uuid: "0a9223a43fab407b8fbe9e5a9536eb99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a9223a43fab407b8fbe9e5a9536eb99" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42259 } } }
I20260810 14:53:29.878296 4198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:29.878304 4199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:29.878593 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.872773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34886 (local address 127.30.74.126:42259)
0810 14:53:29.872959 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:29.872967 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:29.873000 (+ 33us) server_negotiation.cc:407] Connection header received
0810 14:53:29.873474 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:29.873483 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:29.873562 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:29.873682 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:29.873928 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:29.874839 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:29.876007 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:29.876531 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:29.876627 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:29.877415 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:29.877441 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:29.877441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:53:29.877661 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:29.877701 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:53:29.877717 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:29.878022 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:29.878049 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 14:53:29.878159 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260810 14:53:29.879201 4208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:29.879657 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:53:29.879725 4208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:29.881350 4208 catalog_manager.cc:1347] Generated new cluster ID: 95be922c0793474c8d545fc86ee6d901
I20260810 14:53:29.881403 4208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:53:29.894550 4208 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:29.895272 4208 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:29.907573 4208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a9223a43fab407b8fbe9e5a9536eb99: Generated new TSK 0
I20260810 14:53:29.907815 4208 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 14:53:29.930426 4226 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 14:53:29.930528 4226 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 14:53:29.930888 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:29.916084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34888 (local address 127.30.74.126:42259)
0810 14:53:29.916135 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:29.916137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:29.916216 (+ 79us) server_negotiation.cc:407] Connection header received
0810 14:53:29.916277 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:29.916278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:29.916291 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:29.916382 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0810 14:53:29.916550 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:29.917363 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:29.918264 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:29.918590 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:29.918665 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:29.918837 (+ 172us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 14:53:29.930635 (+ 11798us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 14:53:29.930803 (+ 168us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34888: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":27}
W20260810 14:53:29.930950 4196 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34888: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 14:53:29.930985 4225 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:29.915986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42259 (local address 127.0.0.1:34888)
0810 14:53:29.916131 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:29.916142 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:29.916209 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:29.916391 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:29.916391 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:29.916393 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 14:53:29.916476 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:29.916479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:29.917478 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:29.917479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:29.918148 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:29.918151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:29.918674 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:29.918675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:29.918720 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:29.918726 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 14:53:29.930775 (+ 12049us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 14:53:29.930883 (+ 108us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.74.126:42259: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43239/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260810 14:53:29.931313 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 4121
2026-08-10T14:53:29Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (234 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 14:53:29.971913 31017 mini_oidc.cc:152] Starting JWKS server
I20260810 14:53:29.972726 31017 webserver.cc:460] Webserver started at https://127.0.0.1:33273/ using document root <none> and password file <none>
I20260810 14:53:29.972805 31017 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 14:53:29.972973 31017 webserver.cc:460] Webserver started at http://127.0.0.1:36121/ using document root <none> and password file <none>
2026-08-10T14:53:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:29Z Disabled control of system clock
I20260810 14:53:29.982242 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:43855
--webserver_interface=127.30.74.126
--webserver_port=0
--builtin_ntp_servers=127.30.74.84:45877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:43855
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33273/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 14:53:30.065364 4232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:30.065589 4232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:30.065613 4232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:30.066883 4232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:30.066931 4232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:30.066944 4232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:30.066957 4232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:30.066970 4232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 14:53:30.066982 4232 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 14:53:30.066994 4232 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33273/jwks.json
I20260810 14:53:30.068454 4232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:43855
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:43855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33273/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:30.068667 4232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:30.068914 4232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:30.073925 4238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:30.073941 4237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:30.073941 4240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:30.074128 4232 server_base.cc:1034] running on GCE node
I20260810 14:53:30.074466 4232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:30.074779 4232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:30.075935 4232 hybrid_clock.cc:648] HybridClock initialized: now 1786373610075918 us; error 33 us; skew 500 ppm
I20260810 14:53:30.077298 4232 webserver.cc:460] Webserver started at http://127.30.74.126:41625/ using document root <none> and password file <none>
I20260810 14:53:30.077541 4232 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:30.077590 4232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:30.077728 4232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:30.078812 4232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "60b312118f414efe9b3332ce222d52ad"
format_stamp: "Formatted at 2026-08-10 14:53:30 on dist-test-slave-0b58"
server_key: "665838371eb455d4de02dfcf42d2d921"
server_key_iv: "a96c747b3cbc9385c8bfa50121331ea1"
server_key_version: "encryptionkey@0"
I20260810 14:53:30.079180 4232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "60b312118f414efe9b3332ce222d52ad"
format_stamp: "Formatted at 2026-08-10 14:53:30 on dist-test-slave-0b58"
server_key: "665838371eb455d4de02dfcf42d2d921"
server_key_iv: "a96c747b3cbc9385c8bfa50121331ea1"
server_key_version: "encryptionkey@0"
I20260810 14:53:30.080533 4232 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:53:30.081295 4246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:30.081516 4232 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:30.081594 4232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "60b312118f414efe9b3332ce222d52ad"
format_stamp: "Formatted at 2026-08-10 14:53:30 on dist-test-slave-0b58"
server_key: "665838371eb455d4de02dfcf42d2d921"
server_key_iv: "a96c747b3cbc9385c8bfa50121331ea1"
server_key_version: "encryptionkey@0"
I20260810 14:53:30.081681 4232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:30.093894 4232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:30.094146 4232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:30.094252 4232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:30.098155 4232 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:43855
I20260810 14:53:30.098253 4300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:43855 every 8 connection(s)
I20260810 14:53:30.098559 4232 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:30.099248 4301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:30.102046 4301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad: Bootstrap starting.
I20260810 14:53:30.102653 4301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:30.102938 4301 log.cc:824] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:30.103677 4301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad: No bootstrap required, opened a new log
I20260810 14:53:30.105337 4301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b312118f414efe9b3332ce222d52ad" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43855 } }
I20260810 14:53:30.105499 4301 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:30.105546 4301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60b312118f414efe9b3332ce222d52ad, State: Initialized, Role: FOLLOWER
I20260810 14:53:30.105715 4301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b312118f414efe9b3332ce222d52ad" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43855 } }
I20260810 14:53:30.105813 4301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:30.105857 4301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:30.105911 4301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:30.106467 4301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b312118f414efe9b3332ce222d52ad" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43855 } }
I20260810 14:53:30.106586 4301 leader_election.cc:302] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60b312118f414efe9b3332ce222d52ad; no voters:
I20260810 14:53:30.106765 4301 leader_election.cc:288] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:30.106812 4304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:30.107033 4304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [term 1 LEADER]: Becoming Leader. State: Replica: 60b312118f414efe9b3332ce222d52ad, State: Running, Role: LEADER
I20260810 14:53:30.107105 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 4232
I20260810 14:53:30.107143 4304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b312118f414efe9b3332ce222d52ad" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43855 } }
I20260810 14:53:30.107218 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:30.107388 31017 external_mini_cluster.cc:1428] Setting key 4c72121d349e7ffef428f5e568f8f30b
I20260810 14:53:30.107568 4305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60b312118f414efe9b3332ce222d52ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b312118f414efe9b3332ce222d52ad" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43855 } } }
I20260810 14:53:30.107663 4305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:30.107698 4301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:30.107944 4309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:30.108304 4307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60b312118f414efe9b3332ce222d52ad. Latest consensus state: current_term: 1 leader_uuid: "60b312118f414efe9b3332ce222d52ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60b312118f414efe9b3332ce222d52ad" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43855 } } }
I20260810 14:53:30.108387 4307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:30.108888 4309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:30.110561 4309 catalog_manager.cc:1347] Generated new cluster ID: 21d338b0f23445fe87a1a5a86411ef81
I20260810 14:53:30.110616 4309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:53:30.114355 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:30.107860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37740 (local address 127.30.74.126:43855)
0810 14:53:30.108233 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:30.108245 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:30.108646 (+ 401us) server_negotiation.cc:407] Connection header received
0810 14:53:30.108896 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:30.108905 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:30.108983 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:30.109417 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:30.109629 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:30.110962 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:30.112061 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:30.112735 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:30.112870 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:30.113597 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:30.113603 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:30.113603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:53:30.113820 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:30.113865 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:53:30.113881 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:30.113954 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:30.114027 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 14:53:30.114110 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":72,"threads_started":1}
I20260810 14:53:30.115373 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:53:30.124150 4309 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:30.124711 4309 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:30.129385 4309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60b312118f414efe9b3332ce222d52ad: Generated new TSK 0
I20260810 14:53:30.129583 4309 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:53:30.142683 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:30.138592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37756 (local address 127.30.74.126:43855)
0810 14:53:30.138653 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:30.138655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:30.138716 (+ 61us) server_negotiation.cc:407] Connection header received
0810 14:53:30.138793 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:30.138794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:30.138807 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:30.138900 (+ 93us) server_negotiation.cc:226] Negotiated authn=JWT
0810 14:53:30.139013 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:30.139935 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:30.140791 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:30.141171 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:30.141245 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:30.141419 (+ 174us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 14:53:30.142447 (+ 1028us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33273/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 14:53:30.142587 (+ 140us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37756: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33273/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 14:53:30.142752 4311 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37756: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33273/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 14:53:30.144601 4334 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:30.138493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43855 (local address 127.0.0.1:37756)
0810 14:53:30.138653 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:30.138664 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:30.138730 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:30.138900 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:30.138900 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:30.138902 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 14:53:30.138948 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:30.138951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:30.140024 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:30.140024 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:30.140702 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:30.140704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:30.141272 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:30.141273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:30.141315 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:30.141320 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 14:53:30.142550 (+ 1230us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33273/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 14:53:30.144504 (+ 1954us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.74.126:43855: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33273/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":1735,"thread_start_us":106,"threads_started":1}
I20260810 14:53:30.144886 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 4232
E20260810 14:53:30.155956 4335 webserver.cc:533] Webserver: sslize failed:
E20260810 14:53:30.156065 4335 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-10T14:53:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (213 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T14:53:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:30Z Disabled control of system clock
I20260810 14:53:30.193668 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:38033
--webserver_interface=127.30.74.126
--webserver_port=0
--builtin_ntp_servers=127.30.74.84:41761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:38033
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/testchainca.cert with env {}
W20260810 14:53:30.278200 4339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:30.278421 4339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:30.278443 4339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:30.279632 4339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:30.279675 4339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:30.279688 4339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:30.279726 4339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:30.279757 4339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 14:53:30.279775 4339 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 14:53:30.279799 4339 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 14:53:30.281337 4339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:41761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:38033
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:38033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:30.281599 4339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:30.281833 4339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:30.286724 4348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:30.286726 4346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:30.286726 4345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:30.286993 4339 server_base.cc:1034] running on GCE node
I20260810 14:53:30.287231 4339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:30.287483 4339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:30.288622 4339 hybrid_clock.cc:648] HybridClock initialized: now 1786373610288601 us; error 41 us; skew 500 ppm
I20260810 14:53:30.289877 4339 webserver.cc:460] Webserver started at http://127.30.74.126:35031/ using document root <none> and password file <none>
I20260810 14:53:30.290113 4339 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:30.290158 4339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:30.290305 4339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:30.291361 4339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "e57c910c64084008b214d08219951142"
format_stamp: "Formatted at 2026-08-10 14:53:30 on dist-test-slave-0b58"
server_key: "147c41c1b82cee630c58a186e5d9e1ba"
server_key_iv: "c3e2f7ef5df093918dff49cd525d42cc"
server_key_version: "encryptionkey@0"
I20260810 14:53:30.291733 4339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "e57c910c64084008b214d08219951142"
format_stamp: "Formatted at 2026-08-10 14:53:30 on dist-test-slave-0b58"
server_key: "147c41c1b82cee630c58a186e5d9e1ba"
server_key_iv: "c3e2f7ef5df093918dff49cd525d42cc"
server_key_version: "encryptionkey@0"
I20260810 14:53:30.293133 4339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:53:30.293877 4354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:30.294068 4339 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:30.294167 4339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "e57c910c64084008b214d08219951142"
format_stamp: "Formatted at 2026-08-10 14:53:30 on dist-test-slave-0b58"
server_key: "147c41c1b82cee630c58a186e5d9e1ba"
server_key_iv: "c3e2f7ef5df093918dff49cd525d42cc"
server_key_version: "encryptionkey@0"
I20260810 14:53:30.294241 4339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:30.304620 4339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:30.304926 4339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:30.305073 4339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:30.308959 4339 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:38033
I20260810 14:53:30.309017 4409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:38033 every 8 connection(s)
I20260810 14:53:30.309350 4339 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:30.310372 4410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:30.312664 4410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142: Bootstrap starting.
I20260810 14:53:30.313268 4410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:30.313508 4410 log.cc:824] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:30.314239 4410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142: No bootstrap required, opened a new log
I20260810 14:53:30.315740 4410 raft_consensus.cc:348] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57c910c64084008b214d08219951142" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 38033 } }
I20260810 14:53:30.315903 4410 raft_consensus.cc:374] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:30.315927 4410 raft_consensus.cc:729] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e57c910c64084008b214d08219951142, State: Initialized, Role: FOLLOWER
I20260810 14:53:30.316068 4410 consensus_queue.cc:260] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57c910c64084008b214d08219951142" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 38033 } }
I20260810 14:53:30.316149 4410 raft_consensus.cc:388] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:30.316172 4410 raft_consensus.cc:482] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:30.316201 4410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:30.316812 4410 raft_consensus.cc:504] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57c910c64084008b214d08219951142" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 38033 } }
I20260810 14:53:30.316901 4410 leader_election.cc:302] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e57c910c64084008b214d08219951142; no voters:
I20260810 14:53:30.317023 4410 leader_election.cc:288] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:30.317108 4413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:30.317278 4413 raft_consensus.cc:686] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [term 1 LEADER]: Becoming Leader. State: Replica: e57c910c64084008b214d08219951142, State: Running, Role: LEADER
I20260810 14:53:30.317344 4410 sys_catalog.cc:564] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:30.317389 4413 consensus_queue.cc:237] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57c910c64084008b214d08219951142" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 38033 } }
I20260810 14:53:30.317745 4415 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e57c910c64084008b214d08219951142. Latest consensus state: current_term: 1 leader_uuid: "e57c910c64084008b214d08219951142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57c910c64084008b214d08219951142" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 38033 } } }
I20260810 14:53:30.317894 4414 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e57c910c64084008b214d08219951142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57c910c64084008b214d08219951142" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 38033 } } }
I20260810 14:53:30.317901 4415 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:30.318002 4414 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:30.318171 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 4339
I20260810 14:53:30.318185 4422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:30.318293 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:30.318512 31017 external_mini_cluster.cc:1428] Setting key 3e566beb9206c44926728baccff3cb90
I20260810 14:53:30.318714 4422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:30.321316 4422 catalog_manager.cc:1347] Generated new cluster ID: 25f62968fc0946b8832c7d3f069b7792
I20260810 14:53:30.321382 4422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:53:30.324402 4426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:30.319247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59576 (local address 127.30.74.126:38033)
0810 14:53:30.319466 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:30.319476 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:30.319509 (+ 33us) server_negotiation.cc:407] Connection header received
0810 14:53:30.319631 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:30.319638 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:30.319693 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:30.319850 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:30.320127 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:30.321446 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:30.322237 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:30.322697 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:30.322845 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:30.323579 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:30.323583 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:30.323583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:53:30.323773 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:30.323806 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:53:30.323818 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:30.323897 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:30.323983 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 14:53:30.324168 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260810 14:53:30.325331 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:53:30.341675 4422 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:30.342228 4422 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:30.350724 4422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e57c910c64084008b214d08219951142: Generated new TSK 0
I20260810 14:53:30.350975 4422 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:53:30.384322 4426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:30.363246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59586 (local address 127.30.74.126:38033)
0810 14:53:30.363318 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:30.363321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:30.363432 (+ 111us) server_negotiation.cc:407] Connection header received
0810 14:53:30.363518 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:30.363519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:30.363530 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:30.363595 (+ 65us) server_negotiation.cc:226] Negotiated authn=JWT
0810 14:53:30.363757 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:30.364322 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:30.365492 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:30.365684 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:30.365731 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:30.365938 (+ 207us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 14:53:30.384016 (+ 18078us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 14:53:30.384217 (+ 201us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59586: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 14:53:30.384408 4426 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59586: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 14:53:30.384609 4442 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:30.363172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:38033 (local address 127.0.0.1:59586)
0810 14:53:30.363353 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:30.363366 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:30.363441 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:30.363617 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:30.363618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:30.363621 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0810 14:53:30.363688 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:30.363691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:30.364414 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:30.364415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:30.365395 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:30.365399 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:30.365767 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:30.365768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:30.365835 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:30.365841 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 14:53:30.384223 (+ 18382us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 14:53:30.384483 (+ 260us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.74.126:38033: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":119,"threads_started":1}
I20260810 14:53:30.384999 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 4339
2026-08-10T14:53:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (218 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (92 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (92 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:53:30 dist-test-slave-0b58 krb5kdc[4454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:53:30 dist-test-slave-0b58 krb5kdc[4454](info): set up 2 sockets
Aug 10 14:53:30 dist-test-slave-0b58 krb5kdc[4454](info): commencing operation
krb5kdc: starting...
W20260810 14:53:31.656438 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:53:31 dist-test-slave-0b58 krb5kdc[4454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:53:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:53:31.787470 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:42875
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:46773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:42875
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:31.873222 4468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:31.873512 4468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:31.873557 4468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:31.874836 4468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:31.874924 4468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:31.874948 4468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:31.874967 4468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:31.875023 4468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:31.876662 4468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:46773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:42875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:42875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:31.876969 4468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:31.877240 4468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:31.882596 4474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:31.882627 4473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:31.882638 4476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:31.882676 4468 server_base.cc:1034] running on GCE node
I20260810 14:53:31.883158 4468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:31.883539 4468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:31.884697 4468 hybrid_clock.cc:648] HybridClock initialized: now 1786373611884658 us; error 58 us; skew 500 ppm
Aug 10 14:53:31 dist-test-slave-0b58 krb5kdc[4454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373611, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:31.892505 4468 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:53:31.893071 4468 webserver.cc:460] Webserver started at http://127.30.74.126:39189/ using document root <none> and password file <none>
I20260810 14:53:31.893374 4468 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:31.893431 4468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:31.893537 4468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:31.894692 4468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "acd2b33416cb493b888ac9413113a06d"
format_stamp: "Formatted at 2026-08-10 14:53:31 on dist-test-slave-0b58"
server_key: "0c22d2b4c2aa8a8f943e17765f88f558"
server_key_iv: "37a70cf192d4d2072c963a72b6b92dda"
server_key_version: "encryptionkey@0"
I20260810 14:53:31.895041 4468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "acd2b33416cb493b888ac9413113a06d"
format_stamp: "Formatted at 2026-08-10 14:53:31 on dist-test-slave-0b58"
server_key: "0c22d2b4c2aa8a8f943e17765f88f558"
server_key_iv: "37a70cf192d4d2072c963a72b6b92dda"
server_key_version: "encryptionkey@0"
I20260810 14:53:31.896566 4468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:53:31.897445 4483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:31.897636 4468 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:31.897723 4468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "acd2b33416cb493b888ac9413113a06d"
format_stamp: "Formatted at 2026-08-10 14:53:31 on dist-test-slave-0b58"
server_key: "0c22d2b4c2aa8a8f943e17765f88f558"
server_key_iv: "37a70cf192d4d2072c963a72b6b92dda"
server_key_version: "encryptionkey@0"
I20260810 14:53:31.897807 4468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:31.926807 4468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:31.929396 4468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:31.929539 4468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:31.934216 4468 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:42875
I20260810 14:53:31.934257 4540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:42875 every 8 connection(s)
I20260810 14:53:31.934662 4468 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:31.935665 4541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:31.938426 4541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d: Bootstrap starting.
I20260810 14:53:31.939143 4541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:31.939540 4541 log.cc:824] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:31.940464 4541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d: No bootstrap required, opened a new log
I20260810 14:53:31.942322 4541 raft_consensus.cc:348] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd2b33416cb493b888ac9413113a06d" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42875 } }
I20260810 14:53:31.942485 4541 raft_consensus.cc:374] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:31.942512 4541 raft_consensus.cc:729] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acd2b33416cb493b888ac9413113a06d, State: Initialized, Role: FOLLOWER
I20260810 14:53:31.942700 4541 consensus_queue.cc:260] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd2b33416cb493b888ac9413113a06d" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42875 } }
I20260810 14:53:31.942776 4541 raft_consensus.cc:388] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:31.942844 4541 raft_consensus.cc:482] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:31.942909 4541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:31.943560 4541 raft_consensus.cc:504] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd2b33416cb493b888ac9413113a06d" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42875 } }
I20260810 14:53:31.943666 4541 leader_election.cc:302] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acd2b33416cb493b888ac9413113a06d; no voters:
I20260810 14:53:31.943885 4541 leader_election.cc:288] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:31.944012 4544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:31.944154 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 4468
I20260810 14:53:31.944278 4541 sys_catalog.cc:564] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:31.944335 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:31.944514 4544 raft_consensus.cc:686] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [term 1 LEADER]: Becoming Leader. State: Replica: acd2b33416cb493b888ac9413113a06d, State: Running, Role: LEADER
I20260810 14:53:31.944572 31017 external_mini_cluster.cc:1428] Setting key 2608f89ee880a0a5be143d5c75a2df72
I20260810 14:53:31.944626 4544 consensus_queue.cc:237] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd2b33416cb493b888ac9413113a06d" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42875 } }
I20260810 14:53:31.945025 4545 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acd2b33416cb493b888ac9413113a06d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd2b33416cb493b888ac9413113a06d" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42875 } } }
I20260810 14:53:31.945142 4545 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:31.945037 4546 sys_catalog.cc:455] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [sys.catalog]: SysCatalogTable state changed. Reason: New leader acd2b33416cb493b888ac9413113a06d. Latest consensus state: current_term: 1 leader_uuid: "acd2b33416cb493b888ac9413113a06d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acd2b33416cb493b888ac9413113a06d" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42875 } } }
I20260810 14:53:31.945302 4546 sys_catalog.cc:458] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:31.945411 4555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:31.946020 4555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:31.948207 4555 catalog_manager.cc:1347] Generated new cluster ID: 0464155c8dcd48e89ce0bb2fbb52f272
I20260810 14:53:31.948288 4555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 14:53:31 dist-test-slave-0b58 krb5kdc[4454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:31.956804 4555 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:31.957589 4555 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:31.958673 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:31.945330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57280 (local address 127.30.74.126:42875)
0810 14:53:31.945558 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:31.945571 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:31.945644 (+ 73us) server_negotiation.cc:407] Connection header received
0810 14:53:31.945799 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:31.945808 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:31.945885 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:31.946056 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:31.947024 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:31.948527 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:31.949742 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:31.950460 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:31.950573 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:31.953978 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:31.953985 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:31.953985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:31.954002 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:31.956562 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:31.957095 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:31.957099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:31.957100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:31.957244 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:31.957515 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:31.957516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:31.957517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:31.957926 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:31.958109 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:31.958259 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 14:53:31.958431 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":82,"threads_started":1}
I20260810 14:53:31.963370 4555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acd2b33416cb493b888ac9413113a06d: Generated new TSK 0
I20260810 14:53:31.963624 4555 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:53:31.995743 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42875
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:46773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:32.081650 4569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:32.081897 4569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:32.081940 4569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:32.083149 4569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:32.083228 4569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:32.083276 4569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:53:32.084888 4569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:46773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:32.085160 4569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:32.085407 4569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:32.091750 4575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:32.091720 4577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:32.091744 4574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:32.092396 4569 server_base.cc:1034] running on GCE node
I20260810 14:53:32.092636 4569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:32.092926 4569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:32.094091 4569 hybrid_clock.cc:648] HybridClock initialized: now 1786373612094072 us; error 35 us; skew 500 ppm
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[4454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373612, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:32.096756 4569 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:53:32.097286 4569 webserver.cc:460] Webserver started at http://127.30.74.65:45941/ using document root <none> and password file <none>
I20260810 14:53:32.097538 4569 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:32.097584 4569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:32.097716 4569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:32.098783 4569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "2cd45c5139a9426c9a6f1225d3248fa6"
format_stamp: "Formatted at 2026-08-10 14:53:32 on dist-test-slave-0b58"
server_key: "0c0a20568f22da063f9ee944b5a56f37"
server_key_iv: "894ca00efdfec8959c7719a1d1556e5c"
server_key_version: "encryptionkey@0"
I20260810 14:53:32.099161 4569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "2cd45c5139a9426c9a6f1225d3248fa6"
format_stamp: "Formatted at 2026-08-10 14:53:32 on dist-test-slave-0b58"
server_key: "0c0a20568f22da063f9ee944b5a56f37"
server_key_iv: "894ca00efdfec8959c7719a1d1556e5c"
server_key_version: "encryptionkey@0"
I20260810 14:53:32.100642 4569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 14:53:32.101584 4585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:32.101809 4569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:32.101946 4569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "2cd45c5139a9426c9a6f1225d3248fa6"
format_stamp: "Formatted at 2026-08-10 14:53:32 on dist-test-slave-0b58"
server_key: "0c0a20568f22da063f9ee944b5a56f37"
server_key_iv: "894ca00efdfec8959c7719a1d1556e5c"
server_key_version: "encryptionkey@0"
I20260810 14:53:32.102047 4569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:32.117451 4569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:32.120232 4569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:32.120503 4569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:32.120811 4569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:32.121202 4569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:32.121269 4569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:32.121316 4569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:32.121351 4569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:32.128221 4569 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:40703
I20260810 14:53:32.128326 4706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:40703 every 8 connection(s)
I20260810 14:53:32.128703 4569 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:53:32.132577 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 4569
I20260810 14:53:32.132668 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:53:32.132884 31017 external_mini_cluster.cc:1428] Setting key 26200a7ca508f02c15b4c36e9f8f451d
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[4454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373612, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:32.139879 4709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:32.129508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42875 (local address 127.30.74.65:57603)
0810 14:53:32.129701 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:32.129749 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:32.130770 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:32.131015 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:32.131018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:32.131307 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:32.131523 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:32.131527 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.132217 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.132218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:32.133433 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:32.133436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.133957 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.133958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:32.134074 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:32.134773 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:32.134782 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:32.136396 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:32.138441 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:32.138448 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:32.138450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:32.138694 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:32.138942 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:32.138942 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:32.138943 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:32.139047 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:32.139352 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:32.139357 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:32.139479 (+ 122us) client_negotiation.cc:769] Sending connection context
0810 14:53:32.139563 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 14:53:32.139692 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
I20260810 14:53:32.139878 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:32.129541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:57603 (local address 127.30.74.126:42875)
0810 14:53:32.129619 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:32.129621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:32.129850 (+ 229us) server_negotiation.cc:407] Connection header received
0810 14:53:32.130876 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:32.130877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:32.130887 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:32.130954 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:32.131610 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.132113 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.133553 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.133865 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.133942 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:32.136480 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:32.136482 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:32.136482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:32.136491 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:32.138334 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:32.138796 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:32.138797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:32.138798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:32.138865 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:32.139114 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:32.139115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:32.139116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:32.139279 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:32.139363 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:32.139617 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 14:53:32.139726 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 14:53:32.140501 4707 heartbeater.cc:347] Connected to a master server at 127.30.74.126:42875
I20260810 14:53:32.140621 4707 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:32.140843 4707 heartbeater.cc:511] Master 127.30.74.126:42875 requested a full tablet report, sending...
I20260810 14:53:32.141443 4502 ts_manager.cc:194] Registered new tserver with Master: 2cd45c5139a9426c9a6f1225d3248fa6 (127.30.74.65:40703)
I20260810 14:53:32.142074 4502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:57603
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:53:32.164515 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42875
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:46773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:32.255514 4712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:32.255776 4712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:32.255820 4712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:32.257063 4712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:32.257135 4712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:32.257184 4712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:53:32.258764 4712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:46773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:32.259033 4712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:32.259287 4712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:32.264570 4717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:32.264575 4719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:32.264570 4721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:32.264928 4712 server_base.cc:1034] running on GCE node
I20260810 14:53:32.265136 4712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:32.265368 4712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:32.266515 4712 hybrid_clock.cc:648] HybridClock initialized: now 1786373612266502 us; error 29 us; skew 500 ppm
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[4454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373612, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:32.269153 4712 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:53:32.269567 4712 webserver.cc:460] Webserver started at http://127.30.74.66:33575/ using document root <none> and password file <none>
I20260810 14:53:32.269778 4712 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:32.269824 4712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:32.269915 4712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:32.271050 4712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "127c07d2a34947a28fc79d47ab78e381"
format_stamp: "Formatted at 2026-08-10 14:53:32 on dist-test-slave-0b58"
server_key: "27ee229eedad18902892826903b4afdc"
server_key_iv: "8a87f92218c94cd80d9b9ec656538d78"
server_key_version: "encryptionkey@0"
I20260810 14:53:32.271432 4712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "127c07d2a34947a28fc79d47ab78e381"
format_stamp: "Formatted at 2026-08-10 14:53:32 on dist-test-slave-0b58"
server_key: "27ee229eedad18902892826903b4afdc"
server_key_iv: "8a87f92218c94cd80d9b9ec656538d78"
server_key_version: "encryptionkey@0"
I20260810 14:53:32.272891 4712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260810 14:53:32.273731 4728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:32.274101 4712 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:32.274192 4712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "127c07d2a34947a28fc79d47ab78e381"
format_stamp: "Formatted at 2026-08-10 14:53:32 on dist-test-slave-0b58"
server_key: "27ee229eedad18902892826903b4afdc"
server_key_iv: "8a87f92218c94cd80d9b9ec656538d78"
server_key_version: "encryptionkey@0"
I20260810 14:53:32.274290 4712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:32.293972 4712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:32.296582 4712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:32.296751 4712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:32.297030 4712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:32.297391 4712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:32.297453 4712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:32.297501 4712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:32.297526 4712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:32.304035 4712 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:35897
I20260810 14:53:32.304134 4848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:35897 every 8 connection(s)
I20260810 14:53:32.304548 4712 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:53:32.310658 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 4712
I20260810 14:53:32.310751 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[4454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373612, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:32.310956 31017 external_mini_cluster.cc:1428] Setting key 0dc408b4c78732ba02b8a843299e85f6
I20260810 14:53:32.314838 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:32.305400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:57291 (local address 127.30.74.126:42875)
0810 14:53:32.305476 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:32.305478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:32.305671 (+ 193us) server_negotiation.cc:407] Connection header received
0810 14:53:32.306735 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:32.306736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:32.306749 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:32.306814 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:32.307462 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.307970 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.308987 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.309176 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.309239 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:32.311530 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:32.311531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:32.311531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:32.311537 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:32.313136 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:32.313692 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:32.313694 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:32.313694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:32.313775 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:32.314084 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:32.314085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:32.314085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:32.314240 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:32.314315 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:32.314580 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 14:53:32.314680 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 14:53:32.314868 4851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:32.305329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42875 (local address 127.30.74.66:57291)
0810 14:53:32.305538 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:32.305582 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:32.306618 (+ 1036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:32.306889 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:32.306893 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:32.307152 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:32.307379 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:32.307383 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.308049 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.308050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:32.308912 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:32.308914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.309272 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.309273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:32.309381 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:32.309877 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:32.309885 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:32.311450 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:32.313230 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:32.313238 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:32.313240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:32.313591 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:32.313879 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:32.313880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:32.313880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:32.314001 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:32.314313 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:32.314318 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:32.314445 (+ 127us) client_negotiation.cc:769] Sending connection context
0810 14:53:32.314535 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 14:53:32.314695 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":112,"threads_started":1}
I20260810 14:53:32.315466 4849 heartbeater.cc:347] Connected to a master server at 127.30.74.126:42875
I20260810 14:53:32.315585 4849 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:32.315817 4849 heartbeater.cc:511] Master 127.30.74.126:42875 requested a full tablet report, sending...
I20260810 14:53:32.316306 4502 ts_manager.cc:194] Registered new tserver with Master: 127c07d2a34947a28fc79d47ab78e381 (127.30.74.66:35897)
I20260810 14:53:32.316880 4502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:57291
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:53:32.339561 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42875
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:46773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:32.425756 4854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:32.425992 4854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:32.426021 4854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:32.427343 4854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:32.427397 4854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:32.427425 4854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:53:32.429101 4854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:46773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:32.429392 4854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:32.429653 4854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:32.435047 4863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:32.435025 4860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:32.435088 4854 server_base.cc:1034] running on GCE node
W20260810 14:53:32.435025 4861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:32.435472 4854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:32.435736 4854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:32.436880 4854 hybrid_clock.cc:648] HybridClock initialized: now 1786373612436871 us; error 35 us; skew 500 ppm
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[4454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373612, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:32.439634 4854 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:53:32.440107 4854 webserver.cc:460] Webserver started at http://127.30.74.67:33395/ using document root <none> and password file <none>
I20260810 14:53:32.440419 4854 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:32.440490 4854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:32.440653 4854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:32.441764 4854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "59a86818c50d41dd88956723645f231f"
format_stamp: "Formatted at 2026-08-10 14:53:32 on dist-test-slave-0b58"
server_key: "4a8a36bb8dd28624b34e03b8d44dad36"
server_key_iv: "ec76aad2ed114e6b40749cd6d2919e63"
server_key_version: "encryptionkey@0"
I20260810 14:53:32.442152 4854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "59a86818c50d41dd88956723645f231f"
format_stamp: "Formatted at 2026-08-10 14:53:32 on dist-test-slave-0b58"
server_key: "4a8a36bb8dd28624b34e03b8d44dad36"
server_key_iv: "ec76aad2ed114e6b40749cd6d2919e63"
server_key_version: "encryptionkey@0"
I20260810 14:53:32.443567 4854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:53:32.444455 4870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:32.444650 4854 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:32.444769 4854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "59a86818c50d41dd88956723645f231f"
format_stamp: "Formatted at 2026-08-10 14:53:32 on dist-test-slave-0b58"
server_key: "4a8a36bb8dd28624b34e03b8d44dad36"
server_key_iv: "ec76aad2ed114e6b40749cd6d2919e63"
server_key_version: "encryptionkey@0"
I20260810 14:53:32.444867 4854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:32.468357 4854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:32.470372 4854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:32.470546 4854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:32.470805 4854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:32.471148 4854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:32.471205 4854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:32.471252 4854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:32.471283 4854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:32.478143 4854 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:43453
I20260810 14:53:32.478188 4990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:43453 every 8 connection(s)
I20260810 14:53:32.478542 4854 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[4454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373612, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:32.485817 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 4854
I20260810 14:53:32.485899 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:53:32.486142 31017 external_mini_cluster.cc:1428] Setting key 60a01c91a7f8ac0e99642992fe67871c
I20260810 14:53:32.489040 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:32.479298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:44643 (local address 127.30.74.126:42875)
0810 14:53:32.479345 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:32.479347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:32.479633 (+ 286us) server_negotiation.cc:407] Connection header received
0810 14:53:32.480753 (+ 1120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:32.480755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:32.480770 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:32.480824 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:32.481503 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.482023 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.483084 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.483288 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.483367 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:32.485831 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:32.485833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:32.485834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:32.485850 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:32.487548 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:32.487972 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:32.487973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:32.487973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:32.488029 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:32.488450 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:32.488451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:32.488451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:32.488570 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:32.488627 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:32.488901 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 14:53:32.488975 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:53:32.489238 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:32.479300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42875 (local address 127.30.74.67:44643)
0810 14:53:32.479494 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:32.479535 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:32.480627 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:32.480903 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:32.480907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:32.481193 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:32.481396 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:32.481401 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.482134 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.482135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:32.482977 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:32.482980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.483379 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.483380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:32.483521 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:32.484072 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:32.484081 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:32.485707 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:32.487643 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:32.487651 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:32.487653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:32.487899 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:32.488166 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:32.488167 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:32.488168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:32.488384 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:32.488636 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:32.488642 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:32.488775 (+ 133us) client_negotiation.cc:769] Sending connection context
0810 14:53:32.488859 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 14:53:32.489000 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":93,"threads_started":1}
I20260810 14:53:32.489732 4993 heartbeater.cc:347] Connected to a master server at 127.30.74.126:42875
I20260810 14:53:32.489842 4993 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:32.490000 4993 heartbeater.cc:511] Master 127.30.74.126:42875 requested a full tablet report, sending...
I20260810 14:53:32.490386 4502 ts_manager.cc:194] Registered new tserver with Master: 59a86818c50d41dd88956723645f231f (127.30.74.67:43453)
I20260810 14:53:32.490795 4502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:44643
I20260810 14:53:32.498052 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/test-admin.keytab.
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[4454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:32.515149 31017 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 14:53:32.516999 5000 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:32.516151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52916 (local address 127.30.74.65:40703)
0810 14:53:32.516372 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:32.516377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:32.516388 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:53:32.516414 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:32.516416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:32.516430 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:32.516504 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:32.516886 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52916: BlockingRecv error: recv got EOF from 127.0.0.1:52916 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":94,"threads_started":1}
W20260810 14:53:32.517098 31017 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/krb5cc' of type FILE
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[4454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:32.518173 31017 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[4454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:32.519117 31017 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[4454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.65@KRBTEST.COM
I20260810 14:53:32.527812 5000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:32.519683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52926 (local address 127.30.74.65:40703)
0810 14:53:32.519768 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:32.519770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:32.519818 (+ 48us) server_negotiation.cc:407] Connection header received
0810 14:53:32.519922 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:32.519923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:32.519934 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:32.520007 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:32.520491 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.521222 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.522007 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.522381 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.522480 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:32.524966 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:32.524976 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:32.524976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:32.524985 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:32.526596 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:32.526964 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:32.526968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:32.526968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:32.527032 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:32.527287 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:32.527288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:32.527288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:32.527449 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:32.527579 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:32.527644 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 14:53:32.527722 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 14:53:32.529892 5000 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:32.529295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52932 (local address 127.30.74.65:40703)
0810 14:53:32.529372 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:32.529373 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:32.529423 (+ 50us) server_negotiation.cc:407] Connection header received
0810 14:53:32.529478 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:32.529479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:32.529488 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:32.529534 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:32.529841 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52932: BlockingRecv error: recv got EOF from 127.0.0.1:52932 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260810 14:53:32.530085 31017 init.cc:244] Error while opening credential cache '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/krb5cc)
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[4454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:32.531219 31017 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[4454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:32.532114 31017 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[4454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.65@KRBTEST.COM
I20260810 14:53:32.538947 5000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:32.532705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52948 (local address 127.30.74.65:40703)
0810 14:53:32.532739 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:32.532741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:32.532804 (+ 63us) server_negotiation.cc:407] Connection header received
0810 14:53:32.532854 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:32.532854 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:32.532864 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:32.532910 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:32.533257 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.533725 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.534399 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.534591 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.534642 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:32.536675 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:32.536677 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:32.536677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:32.536683 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:32.537909 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:32.538220 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:32.538221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:32.538221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:32.538274 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:32.538465 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:32.538465 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:32.538466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:32.538562 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:32.538658 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:32.538816 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 14:53:32.538883 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260810 14:53:32.540648 5000 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:32.540002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52964 (local address 127.30.74.65:40703)
0810 14:53:32.540037 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:32.540040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:32.540111 (+ 71us) server_negotiation.cc:407] Connection header received
0810 14:53:32.540192 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:32.540193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:32.540202 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:32.540292 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:32.540576 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52964: BlockingRecv error: recv got EOF from 127.0.0.1:52964 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260810 14:53:32.540887 31017 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373586526129-31017-0/krb5kdc/krb5cc' of type FILE
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[4454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:32.541924 31017 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[4454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:32.542822 31017 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[4454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.65@KRBTEST.COM
I20260810 14:53:32.549981 5000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:32.543356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52972 (local address 127.30.74.65:40703)
0810 14:53:32.543413 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:32.543414 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:32.543448 (+ 34us) server_negotiation.cc:407] Connection header received
0810 14:53:32.543499 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:32.543500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:32.543507 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:32.543554 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:32.543954 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.544582 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.545329 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:32.545517 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:32.545622 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:32.547618 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:32.547619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:32.547619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:32.547625 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:32.548891 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:32.549253 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:32.549253 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:32.549254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:32.549303 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:32.549556 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:32.549557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:32.549558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:32.549676 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:32.549766 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:32.549856 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 14:53:32.549921 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 14:53:32.550705 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 4569
I20260810 14:53:32.557605 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 4712
I20260810 14:53:32.566548 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 4854
I20260810 14:53:32.573747 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 4468
2026-08-10T14:53:32Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2007 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[5014](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[5014](info): set up 2 sockets
Aug 10 14:53:32 dist-test-slave-0b58 krb5kdc[5014](info): commencing operation
krb5kdc: starting...
W20260810 14:53:33.674274 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:53:33 dist-test-slave-0b58 krb5kdc[5014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:53:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:33Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.126.keytab.
Entry for principal oryx/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.126.keytab.
I20260810 14:53:33.812495 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:39103
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.126.keytab
--principal=oryx/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:39103
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:33.900964 5027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:33.901217 5027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:33.901242 5027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:33.902467 5027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:33.902520 5027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:33.902534 5027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:33.902547 5027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:33.902560 5027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:33.904142 5027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:39103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.126.keytab
--principal=oryx/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:39103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:33.904453 5027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:33.904661 5027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:33.910100 5033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:33.910151 5034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:33.910295 5027 server_base.cc:1034] running on GCE node
W20260810 14:53:33.910177 5037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:33.910660 5027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:33.911005 5027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:33.912169 5027 hybrid_clock.cc:648] HybridClock initialized: now 1786373613912140 us; error 53 us; skew 500 ppm
Aug 10 14:53:33 dist-test-slave-0b58 krb5kdc[5014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373613, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:33.915177 5027 init.cc:377] Logged in from keytab as oryx/127.30.74.126@KRBTEST.COM (short username oryx)
I20260810 14:53:33.915724 5027 webserver.cc:460] Webserver started at http://127.30.74.126:32835/ using document root <none> and password file <none>
I20260810 14:53:33.915975 5027 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:33.916047 5027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:33.916214 5027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:33.917399 5027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "687bc504a17c410eb594f13b925e956a"
format_stamp: "Formatted at 2026-08-10 14:53:33 on dist-test-slave-0b58"
server_key: "f8377e4818c3da7ad8c38df4f627f0cc"
server_key_iv: "674b858811714e0038172fea7f29bf73"
server_key_version: "encryptionkey@0"
I20260810 14:53:33.917815 5027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "687bc504a17c410eb594f13b925e956a"
format_stamp: "Formatted at 2026-08-10 14:53:33 on dist-test-slave-0b58"
server_key: "f8377e4818c3da7ad8c38df4f627f0cc"
server_key_iv: "674b858811714e0038172fea7f29bf73"
server_key_version: "encryptionkey@0"
I20260810 14:53:33.919282 5027 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:53:33.920136 5045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:33.920388 5027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:33.920486 5027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "687bc504a17c410eb594f13b925e956a"
format_stamp: "Formatted at 2026-08-10 14:53:33 on dist-test-slave-0b58"
server_key: "f8377e4818c3da7ad8c38df4f627f0cc"
server_key_iv: "674b858811714e0038172fea7f29bf73"
server_key_version: "encryptionkey@0"
I20260810 14:53:33.920591 5027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:33.935866 5027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:33.938079 5027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:33.938318 5027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:33.942610 5027 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:39103
I20260810 14:53:33.942692 5105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:39103 every 8 connection(s)
I20260810 14:53:33.943043 5027 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:33.944111 5106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:33.946483 5106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a: Bootstrap starting.
I20260810 14:53:33.947031 5106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:33.947268 5106 log.cc:824] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:33.947981 5106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a: No bootstrap required, opened a new log
I20260810 14:53:33.949013 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 5027
I20260810 14:53:33.949153 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:33.949360 31017 external_mini_cluster.cc:1428] Setting key d21d546232e9f050f2e9a7dedc0ddae6
I20260810 14:53:33.950031 5106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } }
I20260810 14:53:33.950239 5106 raft_consensus.cc:374] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:33.950282 5106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 687bc504a17c410eb594f13b925e956a, State: Initialized, Role: FOLLOWER
I20260810 14:53:33.950414 5106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } }
I20260810 14:53:33.950492 5106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:33.950532 5106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:33.950587 5106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:33.951848 5106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } }
I20260810 14:53:33.951948 5106 leader_election.cc:302] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 687bc504a17c410eb594f13b925e956a; no voters:
I20260810 14:53:33.952117 5106 leader_election.cc:288] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:33.952145 5111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:33.952277 5111 raft_consensus.cc:686] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [term 1 LEADER]: Becoming Leader. State: Replica: 687bc504a17c410eb594f13b925e956a, State: Running, Role: LEADER
I20260810 14:53:33.952368 5111 consensus_queue.cc:237] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } }
I20260810 14:53:33.952421 5106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:33.952735 5113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } }
I20260810 14:53:33.952986 5113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:33.952919 5112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 687bc504a17c410eb594f13b925e956a. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } }
I20260810 14:53:33.953275 5112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:33.953457 5121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:33.954020 5121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:33.956071 5121 catalog_manager.cc:1347] Generated new cluster ID: 40a0197fd68d4fcd8362e3445970b91c
I20260810 14:53:33.956122 5121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 14:53:33 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM
I20260810 14:53:33.962700 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:33.949979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37530 (local address 127.30.74.126:39103)
0810 14:53:33.950191 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:33.950202 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:33.950249 (+ 47us) server_negotiation.cc:407] Connection header received
0810 14:53:33.950371 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:33.950380 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:33.950439 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:33.950590 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:33.951620 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:33.952966 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:33.954118 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:33.954749 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:33.954858 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:33.958057 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:33.958062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:33.958062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:33.958076 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:33.960277 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:33.960736 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:33.960740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:33.960741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:33.960895 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:33.961172 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:33.961173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:33.961173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:33.961539 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:33.961699 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:33.962299 (+ 600us) server_negotiation.cc:299] Negotiation successful
0810 14:53:33.962452 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":100,"threads_started":1}
I20260810 14:53:33.978296 5121 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:33.979103 5121 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:33.984560 5121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a: Generated new TSK 0
I20260810 14:53:33.984833 5121 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.65.keytab.
Entry for principal oryx/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.65.keytab.
I20260810 14:53:34.019428 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39103
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.65.keytab
--principal=oryx/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:34.121659 5134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:34.121891 5134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:34.121915 5134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:34.123127 5134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:34.123180 5134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:34.123209 5134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:53:34.124963 5134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.65.keytab
--principal=oryx/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:34.125257 5134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:34.125537 5134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:34.131127 5140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:34.131143 5143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:34.131156 5141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:34.131531 5134 server_base.cc:1034] running on GCE node
I20260810 14:53:34.131798 5134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:34.132122 5134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:34.133334 5134 hybrid_clock.cc:648] HybridClock initialized: now 1786373614133317 us; error 26 us; skew 500 ppm
Aug 10 14:53:34 dist-test-slave-0b58 krb5kdc[5014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373614, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:34.136354 5134 init.cc:377] Logged in from keytab as oryx/127.30.74.65@KRBTEST.COM (short username oryx)
I20260810 14:53:34.136942 5134 webserver.cc:460] Webserver started at http://127.30.74.65:40375/ using document root <none> and password file <none>
I20260810 14:53:34.137235 5134 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:34.137326 5134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:34.137506 5134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:34.138767 5134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "dd160ee324f54d4d9d942d1ed6215b31"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "1bed93657bab166fd6d8f97b45b24927"
server_key_iv: "5a2f628eac38593e86e71ab5a1215f0b"
server_key_version: "encryptionkey@0"
I20260810 14:53:34.139204 5134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "dd160ee324f54d4d9d942d1ed6215b31"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "1bed93657bab166fd6d8f97b45b24927"
server_key_iv: "5a2f628eac38593e86e71ab5a1215f0b"
server_key_version: "encryptionkey@0"
I20260810 14:53:34.140993 5134 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260810 14:53:34.142159 5151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:34.142529 5134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:34.142719 5134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "dd160ee324f54d4d9d942d1ed6215b31"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "1bed93657bab166fd6d8f97b45b24927"
server_key_iv: "5a2f628eac38593e86e71ab5a1215f0b"
server_key_version: "encryptionkey@0"
I20260810 14:53:34.142823 5134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:34.162134 5134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:34.164835 5134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:34.165042 5134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:34.165354 5134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:34.165740 5134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:34.165805 5134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:34.165844 5134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:34.165879 5134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:34.172557 5134 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:35259
I20260810 14:53:34.172608 5269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:35259 every 8 connection(s)
I20260810 14:53:34.172997 5134 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:53:34.177246 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 5134
I20260810 14:53:34.177381 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:53:34.177585 31017 external_mini_cluster.cc:1428] Setting key 31c7b94f51813c45fcf2d3516f98630d
Aug 10 14:53:34 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373614, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM
I20260810 14:53:34.184603 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.173845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:58253 (local address 127.30.74.126:39103)
0810 14:53:34.173973 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:34.173981 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:34.174208 (+ 227us) server_negotiation.cc:407] Connection header received
0810 14:53:34.174794 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:34.174795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:34.174812 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:34.174869 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:34.175945 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.176532 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.178008 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.178234 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.178319 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.180711 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:34.180712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:34.180713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:34.180723 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:34.182743 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.183344 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.183346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.183346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.183421 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.183739 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.183741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.183741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.183915 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:34.184001 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:34.184324 (+ 323us) server_negotiation.cc:299] Negotiation successful
0810 14:53:34.184452 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 14:53:34.184671 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.173744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.65:58253)
0810 14:53:34.174014 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:34.174066 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:34.174672 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:34.174948 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:34.174952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:34.175604 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:34.175821 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.175827 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.176643 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.176644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.177886 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.177890 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.178331 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.178332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.178487 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.179097 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:34.179107 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:34.180581 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:34.182860 (+ 2279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.182869 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.182871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.183247 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.183519 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.183520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.183521 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.183655 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.183995 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:34.184001 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:34.184147 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 14:53:34.184239 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 14:53:34.184466 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":104,"threads_started":1}
I20260810 14:53:34.185396 5270 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39103
I20260810 14:53:34.185528 5270 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:34.185761 5270 heartbeater.cc:511] Master 127.30.74.126:39103 requested a full tablet report, sending...
I20260810 14:53:34.186496 5065 ts_manager.cc:194] Registered new tserver with Master: dd160ee324f54d4d9d942d1ed6215b31 (127.30.74.65:35259)
I20260810 14:53:34.187505 5065 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:58253
WARNING: no policy specified for oryx/127.30.74.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.66.keytab.
Entry for principal oryx/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.66.keytab.
I20260810 14:53:34.208707 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39103
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.66.keytab
--principal=oryx/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:34.305202 5276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:34.305473 5276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:34.305517 5276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:34.306759 5276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:34.306834 5276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:34.306882 5276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:53:34.308468 5276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.66.keytab
--principal=oryx/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:34.308756 5276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:34.309006 5276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:34.314153 5282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:34.314275 5276 server_base.cc:1034] running on GCE node
W20260810 14:53:34.314157 5283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:34.314397 5285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:34.314718 5276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:34.315003 5276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:34.316154 5276 hybrid_clock.cc:648] HybridClock initialized: now 1786373614316133 us; error 38 us; skew 500 ppm
Aug 10 14:53:34 dist-test-slave-0b58 krb5kdc[5014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373614, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:34.318773 5276 init.cc:377] Logged in from keytab as oryx/127.30.74.66@KRBTEST.COM (short username oryx)
I20260810 14:53:34.319278 5276 webserver.cc:460] Webserver started at http://127.30.74.66:45929/ using document root <none> and password file <none>
I20260810 14:53:34.319525 5276 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:34.319597 5276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:34.319747 5276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:34.320938 5276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "a7835ad107dd430d838123f68e24a29b"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "747f5d2327b83ead2f3bbcd6e95904ac"
server_key_iv: "e8683f13696446bf8004d1dd47432b60"
server_key_version: "encryptionkey@0"
I20260810 14:53:34.321326 5276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "a7835ad107dd430d838123f68e24a29b"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "747f5d2327b83ead2f3bbcd6e95904ac"
server_key_iv: "e8683f13696446bf8004d1dd47432b60"
server_key_version: "encryptionkey@0"
I20260810 14:53:34.322747 5276 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:53:34.323580 5293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:34.323770 5276 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:34.323889 5276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "a7835ad107dd430d838123f68e24a29b"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "747f5d2327b83ead2f3bbcd6e95904ac"
server_key_iv: "e8683f13696446bf8004d1dd47432b60"
server_key_version: "encryptionkey@0"
I20260810 14:53:34.323979 5276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:34.345050 5276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:34.347769 5276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:34.347978 5276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:34.348361 5276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:34.348726 5276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:34.348788 5276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:34.348831 5276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:34.348860 5276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:34.355820 5276 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:41633
I20260810 14:53:34.355867 5416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:41633 every 8 connection(s)
I20260810 14:53:34.356226 5276 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
Aug 10 14:53:34 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373614, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM
I20260810 14:53:34.365408 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 5276
I20260810 14:53:34.365513 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:53:34.365794 31017 external_mini_cluster.cc:1428] Setting key 5e5577090d921487051196fcc3732e86
I20260810 14:53:34.366346 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.357028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:51551 (local address 127.30.74.126:39103)
0810 14:53:34.357100 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:34.357102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:34.357336 (+ 234us) server_negotiation.cc:407] Connection header received
0810 14:53:34.357890 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:34.357891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:34.357903 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:34.357961 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:34.358970 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.359505 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.360654 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.360868 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.360939 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.363172 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:34.363173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:34.363173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:34.363180 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:34.364758 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.365218 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.365219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.365219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.365291 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.365567 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.365569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.365569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.365729 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:34.365806 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:34.366173 (+ 367us) server_negotiation.cc:299] Negotiation successful
0810 14:53:34.366275 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 14:53:34.366604 5419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.357013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.66:51551)
0810 14:53:34.357203 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:34.357244 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:34.357783 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:34.358021 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:34.358025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:34.358643 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:34.358866 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.358871 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.359595 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.359596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.360527 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.360529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.360966 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.360967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.361092 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.361651 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:34.361660 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:34.363079 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:34.364869 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.364875 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.364877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.365111 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.365364 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.365364 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.365365 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.365469 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.365804 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:34.365810 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:34.366002 (+ 192us) client_negotiation.cc:769] Sending connection context
0810 14:53:34.366138 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 14:53:34.366335 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
I20260810 14:53:34.367120 5417 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39103
I20260810 14:53:34.367236 5417 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:34.367427 5417 heartbeater.cc:511] Master 127.30.74.126:39103 requested a full tablet report, sending...
I20260810 14:53:34.367863 5065 ts_manager.cc:194] Registered new tserver with Master: a7835ad107dd430d838123f68e24a29b (127.30.74.66:41633)
I20260810 14:53:34.368422 5065 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:51551
WARNING: no policy specified for oryx/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.67.keytab.
Entry for principal oryx/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.67.keytab.
I20260810 14:53:34.398926 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39103
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.67.keytab
--principal=oryx/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:34.490235 5422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:34.490499 5422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:34.490545 5422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:34.491758 5422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:34.491832 5422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:34.491880 5422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:53:34.493500 5422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.67.keytab
--principal=oryx/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:34.493808 5422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:34.494057 5422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:34.499567 5431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:34.499748 5429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:34.499625 5427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:34.499917 5422 server_base.cc:1034] running on GCE node
I20260810 14:53:34.500108 5422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:34.500453 5422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:34.501606 5422 hybrid_clock.cc:648] HybridClock initialized: now 1786373614501588 us; error 32 us; skew 500 ppm
Aug 10 14:53:34 dist-test-slave-0b58 krb5kdc[5014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373614, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:34.504417 5422 init.cc:377] Logged in from keytab as oryx/127.30.74.67@KRBTEST.COM (short username oryx)
I20260810 14:53:34.504830 5422 webserver.cc:460] Webserver started at http://127.30.74.67:37711/ using document root <none> and password file <none>
I20260810 14:53:34.505048 5422 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:34.505087 5422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:34.505228 5422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:34.506376 5422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "9562ca92f305486db348c6c91ed30016"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "6b8278dd14270d87856f655ab6b93b18"
server_key_iv: "271ea6f40efce19cbf44b83973df1979"
server_key_version: "encryptionkey@0"
I20260810 14:53:34.506786 5422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "9562ca92f305486db348c6c91ed30016"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "6b8278dd14270d87856f655ab6b93b18"
server_key_iv: "271ea6f40efce19cbf44b83973df1979"
server_key_version: "encryptionkey@0"
I20260810 14:53:34.508401 5422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:53:34.509255 5439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:34.509469 5422 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:34.509577 5422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "9562ca92f305486db348c6c91ed30016"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "6b8278dd14270d87856f655ab6b93b18"
server_key_iv: "271ea6f40efce19cbf44b83973df1979"
server_key_version: "encryptionkey@0"
I20260810 14:53:34.509667 5422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:34.524181 5422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:34.526875 5422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:34.527127 5422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:34.527429 5422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:34.527773 5422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:34.527830 5422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:34.527879 5422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:34.527907 5422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:34.534880 5422 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:36419
I20260810 14:53:34.534925 5558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:36419 every 8 connection(s)
I20260810 14:53:34.535290 5422 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
Aug 10 14:53:34 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373614, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM
I20260810 14:53:34.545100 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 5422
I20260810 14:53:34.545249 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:53:34.545519 31017 external_mini_cluster.cc:1428] Setting key 41a852f73e0d27adaf454f709c931132
I20260810 14:53:34.545730 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.536055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:60149 (local address 127.30.74.126:39103)
0810 14:53:34.536135 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:34.536137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:34.536421 (+ 284us) server_negotiation.cc:407] Connection header received
0810 14:53:34.536977 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:34.536979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:34.536995 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:34.537069 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:34.538082 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.538605 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.539632 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.539831 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.539895 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.542221 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:34.542223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:34.542223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:34.542232 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:34.543899 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.544512 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.544513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.544514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.544583 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.544873 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.544874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.544874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.545010 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:34.545088 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:34.545463 (+ 375us) server_negotiation.cc:299] Negotiation successful
0810 14:53:34.545563 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 14:53:34.545979 5561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.536020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.67:60149)
0810 14:53:34.536234 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:34.536314 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:34.536847 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:34.537148 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:34.537152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:34.537776 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:34.537986 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.537991 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.538694 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.538694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.539524 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.539526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.539917 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.539918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.540025 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.540618 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:34.540626 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:34.542068 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:34.544017 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.544030 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.544033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.544422 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.544661 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.544662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.544662 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.544790 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.545080 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:34.545088 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:34.545270 (+ 182us) client_negotiation.cc:769] Sending connection context
0810 14:53:34.545412 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 14:53:34.545656 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":139,"threads_started":1}
I20260810 14:53:34.546618 5559 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39103
I20260810 14:53:34.546746 5559 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:34.546988 5559 heartbeater.cc:511] Master 127.30.74.126:39103 requested a full tablet report, sending...
I20260810 14:53:34.547427 5064 ts_manager.cc:194] Registered new tserver with Master: 9562ca92f305486db348c6c91ed30016 (127.30.74.67:36419)
I20260810 14:53:34.547894 5064 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:60149
I20260810 14:53:34.557945 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.74.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.74.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.74.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.125.keytab.
Entry for principal oryx/127.30.74.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.125.keytab.
I20260810 14:53:34.604121 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.125:42355
--webserver_interface=127.30.74.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.125.keytab
--principal=oryx/127.30.74.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:39103,127.30.74.125:42355
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:34.691028 5564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:34.691282 5564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:34.691323 5564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:34.692616 5564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:34.692687 5564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:34.692719 5564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:34.692749 5564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:34.692773 5564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:34.694371 5564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:39103,127.30.74.125:42355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.125.keytab
--principal=oryx/127.30.74.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.125:42355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.74.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
W20260810 14:53:34.694564 5564 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.74.126:39103,127.30.74.125:42355'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 14:53:34.694733 5564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:34.694981 5564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:34.700006 5572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:34.700006 5569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:34.700244 5564 server_base.cc:1034] running on GCE node
W20260810 14:53:34.700320 5570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:34.700683 5564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:34.701001 5564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:34.702179 5564 hybrid_clock.cc:648] HybridClock initialized: now 1786373614702165 us; error 35 us; skew 500 ppm
Aug 10 14:53:34 dist-test-slave-0b58 krb5kdc[5014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373614, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:34.704829 5564 init.cc:377] Logged in from keytab as oryx/127.30.74.125@KRBTEST.COM (short username oryx)
I20260810 14:53:34.705322 5564 webserver.cc:460] Webserver started at http://127.30.74.125:40167/ using document root <none> and password file <none>
I20260810 14:53:34.705574 5564 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:34.705628 5564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:34.705778 5564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:34.706926 5564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/data/instance:
uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "136b108eb96505de6d778dbc47cc6c30"
server_key_iv: "3b4335bae91df0cc3624d615e0af58b0"
server_key_version: "encryptionkey@0"
I20260810 14:53:34.707311 5564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/wal/instance:
uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "136b108eb96505de6d778dbc47cc6c30"
server_key_iv: "3b4335bae91df0cc3624d615e0af58b0"
server_key_version: "encryptionkey@0"
I20260810 14:53:34.708997 5564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:53:34.709848 5580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:34.710054 5564 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:34.710144 5564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/wal
uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "136b108eb96505de6d778dbc47cc6c30"
server_key_iv: "3b4335bae91df0cc3624d615e0af58b0"
server_key_version: "encryptionkey@0"
I20260810 14:53:34.710208 5564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:34.736367 5564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:34.738852 5564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:34.738992 5564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:34.743243 5564 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.125:42355
I20260810 14:53:34.743335 5636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.125:42355 every 8 connection(s)
I20260810 14:53:34.743724 5564 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/data/info.pb
I20260810 14:53:34.744889 5637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:34.746552 5637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:53:34.750445 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 5564
Aug 10 14:53:34 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373614, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.125@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM
Aug 10 14:53:34 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.74.125@KRBTEST.COM
I20260810 14:53:34.757200 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.747349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37542 (local address 127.30.74.126:39103)
0810 14:53:34.747424 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:34.747425 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:34.747651 (+ 226us) server_negotiation.cc:407] Connection header received
0810 14:53:34.748093 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:34.748095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:34.748114 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:34.748194 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:34.748895 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.749408 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.750649 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.750977 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.751059 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.753843 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:34.753845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:34.753845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:34.753853 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:34.755607 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.756040 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.756042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.756042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.756120 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.756456 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.756457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.756457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.756626 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:34.756707 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:34.757003 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 14:53:34.757095 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 14:53:34.757406 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.747272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:37542)
0810 14:53:34.747495 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:34.747538 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:34.747998 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:34.748314 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:34.748317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:34.748578 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:34.748779 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.748783 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.749497 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.749498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.750509 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.750512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.751072 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.751073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.751228 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.751939 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:34.751948 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:34.753728 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:34.755699 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.755708 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.755710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.755953 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.756191 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.756192 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.756193 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.756374 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.756697 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:34.756704 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:34.756854 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 14:53:34.756968 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 14:53:34.757134 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":127,"threads_started":1}
I20260810 14:53:34.758436 5637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:53:34.761214 5641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.751005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49944 (local address 127.30.74.125:42355)
0810 14:53:34.751213 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:34.751217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:34.751367 (+ 150us) server_negotiation.cc:407] Connection header received
0810 14:53:34.751396 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:34.751399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:34.751422 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:34.751507 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:34.752335 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.753482 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.754337 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.754845 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.754933 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.757848 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:34.757857 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:34.757857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:34.757865 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:34.759962 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.760362 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.760366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.760366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.760434 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.760644 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.760644 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.760645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.760782 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:34.760907 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:34.761008 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 14:53:34.761068 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":118,"threads_started":1}
Aug 10 14:53:34 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373614, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.125@KRBTEST.COM for oryx/127.30.74.125@KRBTEST.COM
I20260810 14:53:34.766423 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.758831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:42355 (local address 127.0.0.1:49954)
0810 14:53:34.758883 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:34.758898 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:34.758950 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:34.759269 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:34.759270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:34.759421 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:34.759473 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.759475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.760384 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.760385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.761109 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.761111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.761580 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.761581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.761637 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.762056 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:34.762067 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:34.763242 (+ 1175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:34.765506 (+ 2264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.765507 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.765507 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.765686 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.765873 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.765874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.765874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.765926 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.766135 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:34.766136 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:34.766172 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 14:53:34.766251 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 14:53:34.766336 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 14:53:34.766510 5642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.758906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49954 (local address 127.30.74.125:42355)
0810 14:53:34.759107 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:34.759109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:34.759116 (+ 7us) server_negotiation.cc:407] Connection header received
0810 14:53:34.759135 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:34.759136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:34.759145 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:34.759223 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:34.759552 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.760309 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.761190 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.761511 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.761583 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.763330 (+ 1747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:34.763331 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:34.763331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:34.763337 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:34.765384 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.765758 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.765759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.765759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.765813 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.765973 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.765974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.765974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.766072 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:34.766131 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:34.766266 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 14:53:34.766382 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":174,"thread_start_us":113,"threads_started":1}
I20260810 14:53:34.767992 5637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: Bootstrap starting.
I20260810 14:53:34.768734 5637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:34.769024 5637 log.cc:824] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:34.769841 5637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: No bootstrap required, opened a new log
I20260810 14:53:34.771329 5637 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } }
I20260810 14:53:34.771462 5637 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:34.771485 5637 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcc444f3a3e44c90b4aeb42775a3a4f3, State: Initialized, Role: FOLLOWER
I20260810 14:53:34.771647 5637 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } }
I20260810 14:53:34.771977 5637 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:34.771938 5644 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } } }
I20260810 14:53:34.772056 5644 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 14:53:34.773850 5656 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:53:34.773916 5656 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:53:34.775657 5564 master_runner.cc:428] Detected that this master fcc444f3a3e44c90b4aeb42775a3a4f3 is joining an existing cluster
I20260810 14:53:34.775751 5564 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.74.125:42355
I20260810 14:53:34.784137 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.776930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37552 (local address 127.30.74.126:39103)
0810 14:53:34.776993 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:34.776995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:34.777149 (+ 154us) server_negotiation.cc:407] Connection header received
0810 14:53:34.777285 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:34.777286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:34.777300 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:34.777353 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:34.777886 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.778625 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.779442 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.779657 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.779745 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.780889 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:34.780890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:34.780891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:34.780899 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:34.782806 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.783291 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.783293 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.783293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.783356 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.783564 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.783565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.783565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.783704 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:34.783751 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:34.783953 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 14:53:34.784039 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 14:53:34.784168 5662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.776838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:37552)
0810 14:53:34.777067 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:34.777079 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:34.777188 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:34.777367 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:34.777368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:34.777663 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:34.777765 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.777767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.778713 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.778714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.779350 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.779352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.779788 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.779790 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.779848 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.780395 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:34.780400 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:34.780741 (+ 341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:34.782905 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.782907 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.782907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.783197 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.783434 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.783435 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.783436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.783494 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.783791 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:34.783791 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:34.783828 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 14:53:34.783923 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 14:53:34.784037 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":44,"thread_start_us":106,"threads_started":1}
I20260810 14:53:34.784749 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.777101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:42355 (local address 127.0.0.1:49964)
0810 14:53:34.777230 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:34.777240 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:34.777299 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:34.777472 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:34.777473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:34.777694 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:34.777754 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.777755 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.778692 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.778693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.779644 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.779647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.780563 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.780563 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.780625 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.781124 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:34.781130 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:34.781509 (+ 379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:34.783666 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.783668 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.783668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.783900 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.784101 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.784102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.784102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.784153 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.784437 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:34.784438 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:34.784471 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:53:34.784547 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 14:53:34.784654 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":35,"thread_start_us":81,"threads_started":1}
I20260810 14:53:34.784864 5642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.777130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49964 (local address 127.30.74.125:42355)
0810 14:53:34.777171 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:34.777172 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:34.777292 (+ 120us) server_negotiation.cc:407] Connection header received
0810 14:53:34.777370 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:34.777371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:34.777379 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:34.777406 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:34.777835 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.778598 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.779755 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.780444 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.780490 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.781609 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:34.781610 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:34.781611 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:34.781619 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:34.783578 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.783991 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.783992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.783992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.784040 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.784212 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.784212 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.784213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.784353 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:34.784397 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:34.784587 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 14:53:34.784727 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":22}
Aug 10 14:53:34 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373613, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.126@KRBTEST.COM for oryx/127.30.74.125@KRBTEST.COM
I20260810 14:53:34.795437 5642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.787189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49980 (local address 127.30.74.125:42355)
0810 14:53:34.787234 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:34.787236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:34.787455 (+ 219us) server_negotiation.cc:407] Connection header received
0810 14:53:34.787513 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:34.787513 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:34.787532 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:34.787586 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:34.788159 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.788735 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.789925 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.790108 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.790171 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.792636 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:34.792637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:34.792637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:34.792644 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:34.794131 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.794602 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.794603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.794603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.794655 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.794876 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.794876 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.794877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.795016 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:34.795083 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:34.795303 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 14:53:34.795370 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:53:34.795509 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.787140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:42355 (local address 127.0.0.1:49980)
0810 14:53:34.787299 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:34.787354 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:34.787436 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:34.787604 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:34.787607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:34.787978 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:34.788062 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.788065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.788826 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.788827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.789831 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.789834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.790188 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.790189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.790278 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.790803 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:34.790809 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:34.792525 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:34.794222 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.794230 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.794233 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.794522 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.794726 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.794727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.794727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.794809 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.795095 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:34.795103 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:34.795166 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 14:53:34.795262 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 14:53:34.795399 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260810 14:53:34.796331 5064 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.74.125:42355: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3"
member_type: NON_VOTER
last_known_addr {
host: "127.30.74.125"
port: 42355
}
attrs {
promote: true
}
}
dest_uuid: "687bc504a17c410eb594f13b925e956a"
cas_config_opid_index: -1
I20260810 14:53:34.796680 5064 consensus_queue.cc:237] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } }
I20260810 14:53:34.797183 5667 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER fcc444f3a3e44c90b4aeb42775a3a4f3 (127.30.74.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } } }
I20260810 14:53:34.797525 5609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:34.798157 5667 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.74.125:42355
I20260810 14:53:34.798488 5564 master.cc:537] Master@127.30.74.125:42355 shutting down...
I20260810 14:53:34.798451 5665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } } }
I20260810 14:53:34.798513 5609 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 FOLLOWER]: Refusing update from remote peer 687bc504a17c410eb594f13b925e956a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 14:53:34.798532 5665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:34.798883 5667 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 14:53:34.799628 5047 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260810 14:53:34.800374 5047 consensus_peers.cc:487] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a -> Peer fcc444f3a3e44c90b4aeb42775a3a4f3 (127.30.74.125:42355): Couldn't send request to peer fcc444f3a3e44c90b4aeb42775a3a4f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 14:53:34.801297 5564 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:34.801397 5564 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:34.801450 5564 tablet_replica.cc:330] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: stopping tablet replica
I20260810 14:53:34.804088 5564 master.cc:552] Master@127.30.74.125:42355 shutdown complete.
I20260810 14:53:34.804145 5564 master_runner.cc:305] Clearing existing system tablet
I20260810 14:53:34.804342 5564 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:34.804400 5564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:34.805124 5564 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260810 14:53:34.805591 5672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:34.805747 5564 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:34.805810 5564 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:34.805841 5564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/wal
uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "136b108eb96505de6d778dbc47cc6c30"
server_key_iv: "3b4335bae91df0cc3624d615e0af58b0"
server_key_version: "encryptionkey@0"
I20260810 14:53:34.806505 5564 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:34.807849 5564 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 14:53:34.808019 5564 log.cc:1196] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 14:53:34.808296 5564 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: Deleting consensus metadata
I20260810 14:53:34.808432 5564 master_runner.cc:315] Copying system tablet from 127.30.74.126:39103
I20260810 14:53:34.809089 5564 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: tablet copy: Beginning tablet copy session from remote peer at address 127.30.74.126:39103
I20260810 14:53:34.815444 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.809482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37562 (local address 127.30.74.126:39103)
0810 14:53:34.809536 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:34.809538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:34.809626 (+ 88us) server_negotiation.cc:407] Connection header received
0810 14:53:34.809755 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:34.809756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:34.809768 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:34.809826 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:34.810157 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.810691 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.811428 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.811641 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.811720 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.812931 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:34.812932 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:34.812933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:34.812940 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:34.814295 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.814656 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.814657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.814658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.814728 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.814930 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.814931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.814932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.815055 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:34.815128 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:34.815269 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 14:53:34.815351 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 14:53:34.815750 5678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.809418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:37562)
0810 14:53:34.809542 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:34.809554 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:34.809644 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:34.809832 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:34.809833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:34.810020 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:34.810073 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.810075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.810779 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.810781 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.811336 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:34.811339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.811721 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.811722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:34.811840 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.812392 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:34.812397 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:34.812769 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:34.814385 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.814386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.814387 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.814571 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.814796 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:34.814797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:34.814797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:34.814858 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:34.815130 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:34.815131 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:34.815178 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 14:53:34.815243 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 14:53:34.815352 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260810 14:53:34.816154 5086 tablet_copy_service.cc:140] P 687bc504a17c410eb594f13b925e956a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer fcc444f3a3e44c90b4aeb42775a3a4f3 ({username='oryx', principal='oryx/127.30.74.125@KRBTEST.COM'} at 127.0.0.1:37562)
I20260810 14:53:34.816334 5086 tablet_copy_service.cc:161] P 687bc504a17c410eb594f13b925e956a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer fcc444f3a3e44c90b4aeb42775a3a4f3 at {username='oryx', principal='oryx/127.30.74.125@KRBTEST.COM'} at 127.0.0.1:37562: session id = fcc444f3a3e44c90b4aeb42775a3a4f3-00000000000000000000000000000000
I20260810 14:53:34.817180 5086 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 14:53:34.817838 5564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 14:53:34.818830 5564 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: tablet copy: Starting download of 0 data blocks...
I20260810 14:53:34.818986 5564 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: tablet copy: Starting download of 1 WAL segments...
I20260810 14:53:34.819883 5564 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 14:53:34.820832 5086 tablet_copy_service.cc:342] P 687bc504a17c410eb594f13b925e956a: Request end of tablet copy session fcc444f3a3e44c90b4aeb42775a3a4f3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.74.125@KRBTEST.COM'} at 127.0.0.1:37562
I20260810 14:53:34.820896 5086 tablet_copy_service.cc:433] P 687bc504a17c410eb594f13b925e956a: ending tablet copy session fcc444f3a3e44c90b4aeb42775a3a4f3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer fcc444f3a3e44c90b4aeb42775a3a4f3
I20260810 14:53:34.824149 5564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:34.825637 5680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:34.825706 5682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:34.825838 5679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:34.826045 5564 server_base.cc:1034] running on GCE node
I20260810 14:53:34.826215 5564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:34.826382 5564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:34.827548 5564 hybrid_clock.cc:648] HybridClock initialized: now 1786373614827534 us; error 29 us; skew 500 ppm
Aug 10 14:53:34 dist-test-slave-0b58 krb5kdc[5014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373614, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:34.829128 5564 init.cc:377] Logged in from keytab as oryx/127.30.74.125@KRBTEST.COM (short username oryx)
I20260810 14:53:34.829459 5564 webserver.cc:460] Webserver started at http://127.30.74.125:34743/ using document root <none> and password file <none>
I20260810 14:53:34.829602 5564 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:34.829632 5564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:34.830266 5564 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:34.830829 5690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:34.830955 5564 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:34.831004 5564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/wal
uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "136b108eb96505de6d778dbc47cc6c30"
server_key_iv: "3b4335bae91df0cc3624d615e0af58b0"
server_key_version: "encryptionkey@0"
I20260810 14:53:34.831067 5564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:34.845558 5564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:34.847499 5564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:34.850790 5564 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.125:42355
I20260810 14:53:34.851104 5747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.125:42355 every 8 connection(s)
I20260810 14:53:34.851227 5564 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-1/data/info.pb
I20260810 14:53:34.852073 5748 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:53:34.852681 5748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: Bootstrap starting.
I20260810 14:53:34.855160 5748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:53:34.855422 5748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: Bootstrap complete.
I20260810 14:53:34.855674 5748 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } }
I20260810 14:53:34.855749 5748 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: fcc444f3a3e44c90b4aeb42775a3a4f3, State: Initialized, Role: LEARNER
I20260810 14:53:34.855777 5748 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } }
I20260810 14:53:34.855962 5748 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:34.855984 5752 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } } }
I20260810 14:53:34.856094 5752 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: This master's current role is: LEARNER
I20260810 14:53:34.857620 5763 catalog_manager.cc:1259] Loaded cluster ID: 40a0197fd68d4fcd8362e3445970b91c
I20260810 14:53:34.857751 5763 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: loading cluster ID for follower catalog manager: success
I20260810 14:53:34.858971 5763 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: acquiring CA information for follower catalog manager: success
I20260810 14:53:34.859203 5763 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:53:34.869604 5764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:34.862937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50004 (local address 127.30.74.125:42355)
0810 14:53:34.863065 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:34.863067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:34.863077 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:34.863101 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:34.863103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:34.863118 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:34.863193 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:34.863743 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.864500 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.865229 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:34.865444 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:34.865530 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:34.866815 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:34.866819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:34.866819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:34.866832 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:34.868405 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.868753 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.868755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.868755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.868821 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:34.869034 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:34.869035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:34.869035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:34.869149 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:34.869252 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:34.869390 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 14:53:34.869476 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for oryx/127.30.74.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.74.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.74.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.124.keytab.
Entry for principal oryx/127.30.74.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.124.keytab.
I20260810 14:53:34.895505 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.124:34947
--webserver_interface=127.30.74.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.124.keytab
--principal=oryx/127.30.74.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:39103,127.30.74.125:42355,127.30.74.124:34947
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:34.981467 5768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:34.981737 5768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:34.981777 5768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:34.982972 5768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:34.983047 5768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:34.983071 5768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:34.983093 5768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:34.983117 5768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:34.984742 5768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:39103,127.30.74.125:42355,127.30.74.124:34947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.124.keytab
--principal=oryx/127.30.74.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.124:34947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.74.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:34.985028 5768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:34.985276 5768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:34.990569 5773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:34.990576 5775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:34.990809 5768 server_base.cc:1034] running on GCE node
W20260810 14:53:34.990585 5777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:34.991140 5768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:34.991509 5768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:34.992673 5768 hybrid_clock.cc:648] HybridClock initialized: now 1786373614992669 us; error 36 us; skew 500 ppm
Aug 10 14:53:34 dist-test-slave-0b58 krb5kdc[5014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373614, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:34.995290 5768 init.cc:377] Logged in from keytab as oryx/127.30.74.124@KRBTEST.COM (short username oryx)
I20260810 14:53:34.995803 5768 webserver.cc:460] Webserver started at http://127.30.74.124:41911/ using document root <none> and password file <none>
I20260810 14:53:34.995999 5768 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:34.996042 5768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:34.996132 5768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:34.997228 5768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/data/instance:
uuid: "3d4b68df43a641f685798a53d1e6de6f"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "fc6d703e923a80460c0da9cd0ec8a464"
server_key_iv: "55745875fd187ff40adf7238324bedf3"
server_key_version: "encryptionkey@0"
I20260810 14:53:34.997548 5768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/wal/instance:
uuid: "3d4b68df43a641f685798a53d1e6de6f"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "fc6d703e923a80460c0da9cd0ec8a464"
server_key_iv: "55745875fd187ff40adf7238324bedf3"
server_key_version: "encryptionkey@0"
I20260810 14:53:34.998867 5768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:53:34.999866 5784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:35.000113 5768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:35.000232 5768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/wal
uuid: "3d4b68df43a641f685798a53d1e6de6f"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "fc6d703e923a80460c0da9cd0ec8a464"
server_key_iv: "55745875fd187ff40adf7238324bedf3"
server_key_version: "encryptionkey@0"
I20260810 14:53:35.000348 5768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:35.014982 5768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:35.018085 5768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:35.018253 5768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:35.024086 5768 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.124:34947
I20260810 14:53:35.024163 5843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.124:34947 every 8 connection(s)
I20260810 14:53:35.024708 5768 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/data/info.pb
I20260810 14:53:35.025810 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:35.027765 5844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:53:35.031668 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 5768
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373614, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.124@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.74.124@KRBTEST.COM
I20260810 14:53:35.040537 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.028702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37572 (local address 127.30.74.126:39103)
0810 14:53:35.028766 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.028768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.028924 (+ 156us) server_negotiation.cc:407] Connection header received
0810 14:53:35.029638 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.029640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.029657 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.029729 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.030693 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.031570 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.033347 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.033555 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.033654 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.036351 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.036353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.036353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.036362 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.038559 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.039150 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.039152 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.039152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.039228 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.039617 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.039618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.039618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.039789 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.039841 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.040309 (+ 468us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.040415 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 14:53:35.041409 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.028586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:37572)
0810 14:53:35.028780 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.028831 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.029510 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.029805 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.029808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.030234 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.030591 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.030597 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.031683 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.031684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.033217 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.033220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.033682 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.033683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.033831 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.034454 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.034463 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.036196 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.038656 (+ 2460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.038663 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.038665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.039054 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.039320 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.039321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.039322 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.039519 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.039899 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.039906 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.040090 (+ 184us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.040810 (+ 720us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.041111 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260810 14:53:35.042014 5844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:53:35.042990 5847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.032213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43248 (local address 127.30.74.124:34947)
0810 14:53:35.032393 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.032398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.032417 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:53:35.032438 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.032440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.032461 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.032562 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.033385 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.034463 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.035602 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.036124 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.036211 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.039300 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.039309 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.039309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.039317 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.041588 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.042027 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.042030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.042031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.042089 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.042338 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.042338 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.042339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.042512 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.042648 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.042750 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.042862 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":122,"threads_started":1}
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373614, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.124@KRBTEST.COM for oryx/127.30.74.125@KRBTEST.COM
I20260810 14:53:35.050768 5764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.042437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50008 (local address 127.30.74.125:42355)
0810 14:53:35.042501 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.042503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.042554 (+ 51us) server_negotiation.cc:407] Connection header received
0810 14:53:35.042654 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.042655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.042671 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.042718 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.043296 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.044142 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.045331 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.045668 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.045746 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.047966 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.047968 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.047968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.047976 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.049517 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.049894 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.049896 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.049897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.049985 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.050219 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.050220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.050220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.050352 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.050425 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.050593 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.050685 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 14:53:35.050760 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.042324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:42355 (local address 127.0.0.1:50008)
0810 14:53:35.042499 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.042511 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.042579 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.042791 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.042792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.042978 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.043043 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.043045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.044248 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.044249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.045246 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.045249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.045772 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.045773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.045860 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.046371 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.046376 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.047879 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.049624 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.049625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.049625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.049803 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.050074 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.050074 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.050075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.050147 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.050428 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.050429 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.050475 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.050553 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.050659 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260810 14:53:35.051239 5844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373614, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.124@KRBTEST.COM for oryx/127.30.74.124@KRBTEST.COM
I20260810 14:53:35.059067 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.051540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34947 (local address 127.0.0.1:43260)
0810 14:53:35.051637 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.051648 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.051709 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.051828 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.051829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.052001 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.052048 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.052049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.052974 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.052975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.053909 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.053912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.054376 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.054377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.054443 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.054878 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.054883 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.056225 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.058017 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.058018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.058018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.058199 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.058411 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.058412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.058412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.058474 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.058732 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.058733 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.058770 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.058854 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.058952 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 14:53:35.059077 5847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.051591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43260 (local address 127.30.74.124:34947)
0810 14:53:35.051625 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.051627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.051682 (+ 55us) server_negotiation.cc:407] Connection header received
0810 14:53:35.051767 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.051768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.051777 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.051821 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.052119 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.052898 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.054040 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.054288 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.054372 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.056349 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.056351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.056351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.056357 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.057935 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.058284 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.058285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.058285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.058337 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.058545 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.058546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.058546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.058657 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.058727 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.058877 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.059001 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":16}
I20260810 14:53:35.060864 5844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: Bootstrap starting.
I20260810 14:53:35.061664 5844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:35.061910 5844 log.cc:824] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:35.062611 5844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: No bootstrap required, opened a new log
I20260810 14:53:35.064004 5844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } }
I20260810 14:53:35.064129 5844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:35.064152 5844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d4b68df43a641f685798a53d1e6de6f, State: Initialized, Role: FOLLOWER
I20260810 14:53:35.064298 5844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } }
I20260810 14:53:35.064563 5849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } } }
I20260810 14:53:35.064627 5844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:35.064653 5849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: This master's current role is: FOLLOWER
W20260810 14:53:35.066222 5861 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:53:35.066295 5861 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:53:35.068650 5768 master_runner.cc:428] Detected that this master 3d4b68df43a641f685798a53d1e6de6f is joining an existing cluster
I20260810 14:53:35.068706 5768 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.74.124:34947
I20260810 14:53:35.077167 5764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.070262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50018 (local address 127.30.74.125:42355)
0810 14:53:35.070303 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.070305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.070817 (+ 512us) server_negotiation.cc:407] Connection header received
0810 14:53:35.070908 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.070909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.070920 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.070950 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.071334 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.072077 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.073114 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.073438 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.073487 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.074526 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.074527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.074527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.074535 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.075954 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.076364 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.076365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.076366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.076427 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.076629 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.076630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.076630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.076747 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.076833 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.076985 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.077067 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:53:35.077181 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.070344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:42355 (local address 127.0.0.1:50018)
0810 14:53:35.070760 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.070769 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.070841 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.071010 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.071010 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.071188 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.071241 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.071242 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.072167 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.072168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.073014 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.073015 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.073572 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.073573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.073641 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.074092 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.074096 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.074416 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.076059 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.076060 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.076061 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.076293 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.076496 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.076496 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.076497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.076562 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.076824 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.076825 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.076871 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.076959 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.077067 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":64,"threads_started":1}
I20260810 14:53:35.095216 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 5134
I20260810 14:53:35.099809 5868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.069873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34947 (local address 127.0.0.1:43272)
0810 14:53:35.070075 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.070086 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.070166 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.070365 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.070366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.070568 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.070630 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.070632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.071528 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.071529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.072420 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.072422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.072895 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.072896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.072950 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.073522 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.073527 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.096618 (+ 23091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.098807 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.098808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.098808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.099001 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.099168 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.099169 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.099169 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.099227 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.099487 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.099487 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.099521 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.099577 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.099680 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":110,"threads_started":1}
I20260810 14:53:35.100085 5847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.069932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43272 (local address 127.30.74.124:34947)
0810 14:53:35.069971 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.069973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.070143 (+ 170us) server_negotiation.cc:407] Connection header received
0810 14:53:35.070232 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.070233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.070243 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.070302 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.070717 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.071452 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.072500 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.072789 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.072840 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.096914 (+ 24074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.096915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.096915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.096923 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.098715 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.099068 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.099069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.099069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.099111 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.099285 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.099285 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.099286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.099411 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.099467 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.099929 (+ 462us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.099998 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 14:53:35.104936 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.069916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37574 (local address 127.30.74.126:39103)
0810 14:53:35.069965 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.069967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.070234 (+ 267us) server_negotiation.cc:407] Connection header received
0810 14:53:35.070338 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.070339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.070350 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.070381 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.070849 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.071612 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.072756 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.073060 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.073105 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.100343 (+ 27238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.100346 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.100346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.100355 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.103448 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.103934 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.103936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.103936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.104006 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.104278 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.104279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.104280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.104452 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.104523 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.104740 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.104834 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 14:53:35.105010 5869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.070025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:37574)
0810 14:53:35.070152 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.070163 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.070262 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.070446 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.070447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.070655 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.070750 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.070752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.071715 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.071715 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.072644 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.072647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.073186 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.073197 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.073267 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.097383 (+ 24116us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.097388 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.100202 (+ 2814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.103559 (+ 3357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.103560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.103561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.103843 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.104090 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.104091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.104091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.104173 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.104514 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.104515 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.104576 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.104703 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.104840 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260810 14:53:35.107391 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:35259
--local_ip_for_outbound_sockets=127.30.74.65
--tserver_master_addrs=127.30.74.126:39103
--webserver_port=40375
--webserver_interface=127.30.74.65
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.65.keytab
--principal=oryx/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.74.126:39103,127.30.74.125:42355
--tserver_master_addrs=127.30.74.126:39103,127.30.74.125:42355,127.30.74.124:34947 with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373613, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.126@KRBTEST.COM for oryx/127.30.74.124@KRBTEST.COM
I20260810 14:53:35.115741 5847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.107957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43278 (local address 127.30.74.124:34947)
0810 14:53:35.108066 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.108068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.108075 (+ 7us) server_negotiation.cc:407] Connection header received
0810 14:53:35.108107 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.108108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.108118 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.108169 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.108595 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.109294 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.110008 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.110240 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.110309 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.112689 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.112691 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.112691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.112699 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.114412 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.114854 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.114855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.114856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.114917 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.115152 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.115153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.115153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.115302 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.115374 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.115561 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.115642 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 14:53:35.115808 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.107895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34947 (local address 127.0.0.1:43278)
0810 14:53:35.107949 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.107959 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.107994 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.108196 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.108198 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.108443 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.108499 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.108501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.109383 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.109384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.109908 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.109910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.110319 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.110320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.110414 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.110854 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.110860 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.112597 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.114507 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.114509 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.114509 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.114768 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.114998 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.114999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.114999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.115079 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.115382 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.115383 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.115440 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.115524 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.115649 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 14:53:35.116745 5064 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.74.124:34947: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f"
member_type: NON_VOTER
last_known_addr {
host: "127.30.74.124"
port: 34947
}
attrs {
promote: true
}
}
dest_uuid: "687bc504a17c410eb594f13b925e956a"
cas_config_opid_index: 5
I20260810 14:53:35.117041 5064 consensus_queue.cc:237] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } }
I20260810 14:53:35.117475 5667 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3d4b68df43a641f685798a53d1e6de6f (127.30.74.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } } }
I20260810 14:53:35.118588 5667 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.74.124:34947
I20260810 14:53:35.118677 5667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } } }
I20260810 14:53:35.118769 5667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:35.119819 5815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:35.121088 5815 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 FOLLOWER]: Refusing update from remote peer 687bc504a17c410eb594f13b925e956a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 14:53:35.121376 5665 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:35.122686 5815 raft_consensus.cc:858] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } }
W20260810 14:53:35.122771 5815 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } }
I20260810 14:53:35.123549 5815 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER fcc444f3a3e44c90b4aeb42775a3a4f3 (127.30.74.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } } }
I20260810 14:53:35.124611 5768 master.cc:537] Master@127.30.74.124:34947 shutting down...
I20260810 14:53:35.127895 5875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } } }
I20260810 14:53:35.128033 5875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 14:53:35.128083 5875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } } }
I20260810 14:53:35.128135 5875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 14:53:35.128291 5849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 687bc504a17c410eb594f13b925e956a. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } } }
I20260810 14:53:35.128357 5849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 14:53:35.128723 5665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } } }
I20260810 14:53:35.128827 5665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: This master's current role is: LEADER
W20260810 14:53:35.129307 5050 consensus_peers.cc:487] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a -> Peer 3d4b68df43a641f685798a53d1e6de6f (127.30.74.124:34947): Couldn't send request to peer 3d4b68df43a641f685798a53d1e6de6f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 14:53:35.130234 5768 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260810 14:53:35.130352 5768 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260810 14:53:35.130394 5768 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: stopping tablet replica
I20260810 14:53:35.131716 5764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.119168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50026 (local address 127.30.74.125:42355)
0810 14:53:35.123132 (+ 3964us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.123134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.124336 (+ 1202us) server_negotiation.cc:407] Connection header received
0810 14:53:35.124372 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.124373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.124399 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.124477 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:35.127497 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.128392 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.130526 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.131164 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.131216 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.131320 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.131595 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.131642 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3932}
I20260810 14:53:35.132175 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.118998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:42355 (local address 127.0.0.1:50026)
0810 14:53:35.124194 (+ 5196us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.124206 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.124274 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.127319 (+ 3045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.127321 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.127324 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:35.127381 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.127383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.128763 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.128766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.130279 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.130364 (+ 85us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.131417 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.131418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.131488 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.131514 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.131967 (+ 453us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.132095 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5177}
I20260810 14:53:35.132609 5719 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 LEARNER]: Refusing update from remote peer 687bc504a17c410eb594f13b925e956a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 14:53:35.132840 5667 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 14:53:35.133422 5719 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 3d4b68df43a641f685798a53d1e6de6f (127.30.74.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } } }
I20260810 14:53:35.134559 5883 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } } }
I20260810 14:53:35.134668 5883 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: This master's current role is: LEARNER
I20260810 14:53:35.134876 5752 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 687bc504a17c410eb594f13b925e956a. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: NON_VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: true } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } } }
I20260810 14:53:35.134961 5752 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: This master's current role is: LEARNER
I20260810 14:53:35.136790 5667 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a: attempting to promote NON_VOTER fcc444f3a3e44c90b4aeb42775a3a4f3 to VOTER
I20260810 14:53:35.136996 5667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } }
I20260810 14:53:35.137629 5719 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 LEARNER]: Refusing update from remote peer 687bc504a17c410eb594f13b925e956a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 14:53:35.137842 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.137435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34947 (local address 127.0.0.1:43286)
0810 14:53:35.137492 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.137548 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 14:53:35.137852 5884 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 14:53:35.138223 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.138098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34947 (local address 127.0.0.1:43296)
0810 14:53:35.138179 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.138203 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260810 14:53:35.138275 5050 consensus_peers.cc:487] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a -> Peer 3d4b68df43a641f685798a53d1e6de6f (127.30.74.124:34947): Couldn't send request to peer 3d4b68df43a641f685798a53d1e6de6f. Status: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34947: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 14:53:35.138677 5665 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, fcc444f3a3e44c90b4aeb42775a3a4f3 (127.30.74.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } } }
I20260810 14:53:35.139673 5665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } } }
I20260810 14:53:35.139755 5665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:35.139806 5665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } } }
I20260810 14:53:35.139854 5665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:35.140092 5719 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, fcc444f3a3e44c90b4aeb42775a3a4f3 (127.30.74.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } } }
I20260810 14:53:35.141032 5752 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } } }
I20260810 14:53:35.141119 5752 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:53:35.144351 5768 master.cc:552] Master@127.30.74.124:34947 shutdown complete.
I20260810 14:53:35.144411 5768 master_runner.cc:305] Clearing existing system tablet
I20260810 14:53:35.144551 5768 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:35.144610 5768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:35.145442 5768 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:35.146064 5886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:35.146243 5768 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:35.146296 5768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:35.146319 5768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/wal
uuid: "3d4b68df43a641f685798a53d1e6de6f"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "fc6d703e923a80460c0da9cd0ec8a464"
server_key_iv: "55745875fd187ff40adf7238324bedf3"
server_key_version: "encryptionkey@0"
I20260810 14:53:35.147152 5768 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:35.148749 5768 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 14:53:35.148823 5768 log.cc:1196] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 14:53:35.149067 5768 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: Deleting consensus metadata
I20260810 14:53:35.149190 5768 master_runner.cc:315] Copying system tablet from 127.30.74.126:39103
I20260810 14:53:35.149833 5768 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: tablet copy: Beginning tablet copy session from remote peer at address 127.30.74.126:39103
I20260810 14:53:35.160552 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.152700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37590 (local address 127.30.74.126:39103)
0810 14:53:35.152756 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.152758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.152877 (+ 119us) server_negotiation.cc:407] Connection header received
0810 14:53:35.152948 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.152950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.152962 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.153037 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.153418 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.154261 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.155359 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.155670 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.155746 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.157088 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.157089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.157089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.157098 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.159087 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.159549 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.159551 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.159551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.159633 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.159897 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.159898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.159899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.160049 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.160123 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.160364 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.160455 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 14:53:35.160565 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.152639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:37590)
0810 14:53:35.152790 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.152802 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.152876 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.153048 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.153049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.153271 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.153326 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.153329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.154349 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.154350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.155208 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.155211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.155791 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.155793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.155860 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.156458 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.156464 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.156986 (+ 522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.159199 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.159200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.159201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.159447 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.159734 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.159736 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.159736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.159815 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.160143 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.160144 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.160214 (+ 70us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.160328 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.160456 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260810 14:53:35.161355 5086 tablet_copy_service.cc:140] P 687bc504a17c410eb594f13b925e956a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3d4b68df43a641f685798a53d1e6de6f ({username='oryx', principal='oryx/127.30.74.124@KRBTEST.COM'} at 127.0.0.1:37590)
I20260810 14:53:35.161434 5086 tablet_copy_service.cc:161] P 687bc504a17c410eb594f13b925e956a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3d4b68df43a641f685798a53d1e6de6f at {username='oryx', principal='oryx/127.30.74.124@KRBTEST.COM'} at 127.0.0.1:37590: session id = 3d4b68df43a641f685798a53d1e6de6f-00000000000000000000000000000000
I20260810 14:53:35.162117 5086 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 14:53:35.162708 5768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 14:53:35.164193 5768 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: tablet copy: Starting download of 0 data blocks...
I20260810 14:53:35.164462 5768 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: tablet copy: Starting download of 1 WAL segments...
I20260810 14:53:35.165326 5768 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 14:53:35.166339 5086 tablet_copy_service.cc:342] P 687bc504a17c410eb594f13b925e956a: Request end of tablet copy session 3d4b68df43a641f685798a53d1e6de6f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.74.124@KRBTEST.COM'} at 127.0.0.1:37590
I20260810 14:53:35.166399 5086 tablet_copy_service.cc:433] P 687bc504a17c410eb594f13b925e956a: ending tablet copy session 3d4b68df43a641f685798a53d1e6de6f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3d4b68df43a641f685798a53d1e6de6f
I20260810 14:53:35.172042 5768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:35.173264 5897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:35.173444 5894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:35.173537 5895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:35.173988 5768 server_base.cc:1034] running on GCE node
I20260810 14:53:35.174108 5768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:35.174245 5768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:35.175370 5768 hybrid_clock.cc:648] HybridClock initialized: now 1786373615175344 us; error 44 us; skew 500 ppm
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373615, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:35.176959 5768 init.cc:377] Logged in from keytab as oryx/127.30.74.124@KRBTEST.COM (short username oryx)
I20260810 14:53:35.177235 5768 webserver.cc:460] Webserver started at http://127.30.74.124:34219/ using document root <none> and password file <none>
I20260810 14:53:35.177361 5768 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:35.177394 5768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:35.178189 5768 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:35.178824 5904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:35.178982 5768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:35.179039 5768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/wal
uuid: "3d4b68df43a641f685798a53d1e6de6f"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "fc6d703e923a80460c0da9cd0ec8a464"
server_key_iv: "55745875fd187ff40adf7238324bedf3"
server_key_version: "encryptionkey@0"
I20260810 14:53:35.179102 5768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:35.188462 5768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:35.190375 5768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:35.193682 5768 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.124:34947
I20260810 14:53:35.193784 5962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.124:34947 every 8 connection(s)
I20260810 14:53:35.194013 5768 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/master-2/data/info.pb
I20260810 14:53:35.195269 5963 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:53:35.195859 5963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: Bootstrap starting.
I20260810 14:53:35.198511 5963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:53:35.198787 5963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: Bootstrap complete.
I20260810 14:53:35.199023 5963 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } }
I20260810 14:53:35.199136 5963 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3d4b68df43a641f685798a53d1e6de6f, State: Initialized, Role: LEARNER
I20260810 14:53:35.199178 5963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } }
I20260810 14:53:35.199438 5963 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:35.199417 5966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } } }
I20260810 14:53:35.199663 5966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: This master's current role is: LEARNER
I20260810 14:53:35.201222 5978 catalog_manager.cc:1259] Loaded cluster ID: 40a0197fd68d4fcd8362e3445970b91c
I20260810 14:53:35.201316 5978 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: loading cluster ID for follower catalog manager: success
I20260810 14:53:35.202550 5978 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: acquiring CA information for follower catalog manager: success
I20260810 14:53:35.202940 5978 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 14:53:35.219309 5871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:35.219554 5871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:35.219595 5871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:35.220904 5871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:35.220973 5871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:35.221019 5871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:53:35.222644 5871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.65.keytab
--principal=oryx/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:35259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=40375
--tserver_master_addrs=127.30.74.126:39103,127.30.74.125:42355,127.30.74.124:34947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:35.222915 5871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:35.223192 5871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:35.228900 5984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:35.228878 5983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:35.228912 5871 server_base.cc:1034] running on GCE node
W20260810 14:53:35.228878 5986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:35.229373 5871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:35.229673 5871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:35.230815 5871 hybrid_clock.cc:648] HybridClock initialized: now 1786373615230799 us; error 35 us; skew 500 ppm
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373615, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:35.233573 5871 init.cc:377] Logged in from keytab as oryx/127.30.74.65@KRBTEST.COM (short username oryx)
I20260810 14:53:35.234189 5871 webserver.cc:460] Webserver started at http://127.30.74.65:40375/ using document root <none> and password file <none>
I20260810 14:53:35.234423 5871 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:35.234512 5871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:35.235901 5871 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:53:35.236584 5993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:35.236764 5871 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:35.236860 5871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "dd160ee324f54d4d9d942d1ed6215b31"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "1bed93657bab166fd6d8f97b45b24927"
server_key_iv: "5a2f628eac38593e86e71ab5a1215f0b"
server_key_version: "encryptionkey@0"
I20260810 14:53:35.237231 5871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:35.257108 5871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:35.259701 5871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:35.259842 5871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:35.260054 5871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:35.260497 5871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:35.260540 5871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:35.260563 5871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:35.260576 5871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:35.266659 5871 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:35259
I20260810 14:53:35.266746 6119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:35259 every 8 connection(s)
I20260810 14:53:35.267118 5871 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373615, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM
I20260810 14:53:35.274130 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 5871
I20260810 14:53:35.274308 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 5276
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373615, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.125@KRBTEST.COM
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373615, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.124@KRBTEST.COM
I20260810 14:53:35.279994 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.268036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:50711 (local address 127.30.74.126:39103)
0810 14:53:35.268088 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.268091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.268481 (+ 390us) server_negotiation.cc:407] Connection header received
0810 14:53:35.269071 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.269073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.269085 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.269121 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.270090 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.270862 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.271980 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.272225 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.272312 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.276284 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.276286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.276286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.276295 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.278181 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.278666 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.278667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.278668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.278749 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.279063 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.279064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.279064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.279215 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.279254 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.279743 (+ 489us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.279837 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 14:53:35.280793 6125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.267927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.65:50711)
0810 14:53:35.268344 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.268392 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.268943 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.269215 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.269220 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.269742 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.269970 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.269977 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.270957 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.270957 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.271874 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.271877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.272424 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.272425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.272516 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.272910 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.272915 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.276132 (+ 3217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.278278 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.278287 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.278289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.278589 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.278855 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.278855 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.278855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.278985 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.279324 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.279329 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.279590 (+ 261us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.280164 (+ 574us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.280388 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":8,"thread_start_us":119,"threads_started":1}
I20260810 14:53:35.281550 6120 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39103
I20260810 14:53:35.281705 6120 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:35.282002 6120 heartbeater.cc:511] Master 127.30.74.126:39103 requested a full tablet report, sending...
I20260810 14:53:35.282539 5064 ts_manager.cc:194] Re-registered known tserver with Master: dd160ee324f54d4d9d942d1ed6215b31 (127.30.74.65:35259)
I20260810 14:53:35.283089 5064 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:50711
I20260810 14:53:35.286511 5764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.268358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:35745 (local address 127.30.74.125:42355)
0810 14:53:35.268407 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.268409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.268508 (+ 99us) server_negotiation.cc:407] Connection header received
0810 14:53:35.269053 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.269054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.269068 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.269115 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.270072 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.270584 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.271643 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.271856 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.271946 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.283228 (+ 11282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.283230 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.283231 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.283241 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.285278 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.285641 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.285642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.285643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.285701 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.285913 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.285914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.285914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.286064 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.286144 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.286340 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.286416 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 14:53:35.286527 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.268300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:42355 (local address 127.30.74.65:35745)
0810 14:53:35.268454 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.268463 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.268945 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.269483 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.269484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.269754 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.269970 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.269974 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.270720 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.270721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.271545 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.271547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.271959 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.271959 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.272125 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.272770 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.272778 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.283031 (+ 10253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.285372 (+ 2341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.285373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.285373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.285560 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.285780 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.285781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.285781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.285842 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.286176 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.286177 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.286220 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.286300 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.286417 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":79,"threads_started":1}
I20260810 14:53:35.286909 6122 heartbeater.cc:347] Connected to a master server at 127.30.74.125:42355
I20260810 14:53:35.286959 6122 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:35.287038 6122 heartbeater.cc:511] Master 127.30.74.125:42355 requested a full tablet report, sending...
I20260810 14:53:35.287423 5708 ts_manager.cc:194] Registered new tserver with Master: dd160ee324f54d4d9d942d1ed6215b31 (127.30.74.65:35259)
I20260810 14:53:35.287431 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.267905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:32921 (local address 127.30.74.124:34947)
0810 14:53:35.268055 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.268057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.268494 (+ 437us) server_negotiation.cc:407] Connection header received
0810 14:53:35.269069 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.269070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.269086 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.269129 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.270088 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.271125 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.272087 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.272439 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.272531 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.283214 (+ 10683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.283217 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.283217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.283228 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.285856 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.286303 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.286304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.286305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.286369 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.286581 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.286582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.286582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.286747 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.286940 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.287169 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.287282 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
I20260810 14:53:35.287909 6126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.267927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34947 (local address 127.30.74.65:32921)
0810 14:53:35.268433 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.268443 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.268948 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.269215 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.269220 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.269761 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.269970 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.269977 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.271213 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.271214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.272009 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.272010 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.272533 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.272534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.272604 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.273093 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.273098 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.282597 (+ 9499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.285960 (+ 3363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.285961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.285961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.286213 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.286437 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.286438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.286439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.286501 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.287000 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.287000 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.287047 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.287558 (+ 511us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.287725 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":25,"thread_start_us":111,"threads_started":1}
I20260810 14:53:35.288349 6121 heartbeater.cc:347] Connected to a master server at 127.30.74.124:34947
I20260810 14:53:35.288394 6121 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:35.288436 6121 heartbeater.cc:511] Master 127.30.74.124:34947 requested a full tablet report, sending...
I20260810 14:53:35.288775 5924 ts_manager.cc:194] Registered new tserver with Master: dd160ee324f54d4d9d942d1ed6215b31 (127.30.74.65:35259)
I20260810 14:53:35.290856 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:41633
--local_ip_for_outbound_sockets=127.30.74.66
--tserver_master_addrs=127.30.74.126:39103
--webserver_port=45929
--webserver_interface=127.30.74.66
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.66.keytab
--principal=oryx/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.74.126:39103,127.30.74.125:42355
--tserver_master_addrs=127.30.74.126:39103,127.30.74.125:42355,127.30.74.124:34947 with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:35.376853 6128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:35.377074 6128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:35.377100 6128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:35.378407 6128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:35.378453 6128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:35.378481 6128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:53:35.380141 6128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.66.keytab
--principal=oryx/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:41633
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=45929
--tserver_master_addrs=127.30.74.126:39103,127.30.74.125:42355,127.30.74.124:34947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:35.380432 6128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:35.380707 6128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:35.385936 6133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:35.385943 6136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:35.385943 6134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:35.386230 6128 server_base.cc:1034] running on GCE node
I20260810 14:53:35.386466 6128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:35.386729 6128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:35.387903 6128 hybrid_clock.cc:648] HybridClock initialized: now 1786373615387889 us; error 30 us; skew 500 ppm
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373615, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:35.390496 6128 init.cc:377] Logged in from keytab as oryx/127.30.74.66@KRBTEST.COM (short username oryx)
I20260810 14:53:35.390901 6128 webserver.cc:460] Webserver started at http://127.30.74.66:45929/ using document root <none> and password file <none>
I20260810 14:53:35.391129 6128 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:35.391182 6128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:35.392530 6128 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:35.393081 6144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:35.393316 6128 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 14:53:35.393441 6128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "a7835ad107dd430d838123f68e24a29b"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "747f5d2327b83ead2f3bbcd6e95904ac"
server_key_iv: "e8683f13696446bf8004d1dd47432b60"
server_key_version: "encryptionkey@0"
I20260810 14:53:35.393844 6128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:35.407425 6128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:35.409650 6128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:35.409830 6128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:35.410091 6128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:35.410416 6128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:35.410475 6128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:35.410518 6128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:35.410543 6128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:35.416944 6128 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:41633
I20260810 14:53:35.417022 6264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:41633 every 8 connection(s)
I20260810 14:53:35.417402 6128 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373615, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.124@KRBTEST.COM
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373615, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.125@KRBTEST.COM
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373615, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM
I20260810 14:53:35.426604 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 6128
I20260810 14:53:35.426769 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 5422
I20260810 14:53:35.429919 5764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.418563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:43483 (local address 127.30.74.125:42355)
0810 14:53:35.418611 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.418613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.418784 (+ 171us) server_negotiation.cc:407] Connection header received
0810 14:53:35.419669 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.419670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.419687 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.419721 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.420591 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.421371 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.422535 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.422824 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.422869 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.425659 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.425660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.425661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.425670 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.428083 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.428712 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.428713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.428713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.428776 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.429108 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.429109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.429110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.429280 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.429336 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.429732 (+ 396us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.429826 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 14:53:35.430512 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.418168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:42131 (local address 127.30.74.126:39103)
0810 14:53:35.418592 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.418594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.418759 (+ 165us) server_negotiation.cc:407] Connection header received
0810 14:53:35.419673 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.419675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.419689 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.419721 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.420589 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.421350 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.422535 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.422839 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.422875 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.426254 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.426256 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.426256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.426264 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.429148 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.429521 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.429522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.429522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.429582 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.429807 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.429809 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.429809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.429938 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.429985 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.430341 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.430422 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383}
I20260810 14:53:35.430605 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.418494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:42355 (local address 127.30.74.66:43483)
0810 14:53:35.418664 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.418673 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.419566 (+ 893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.419832 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.419833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.420276 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.420471 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.420478 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.421461 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.421461 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.422429 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.422432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.422955 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.422956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.423029 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.423602 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.423606 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.425532 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.428210 (+ 2678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.428222 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.428225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.428606 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.428856 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.428857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.428857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.429020 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.429372 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.429380 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.429566 (+ 186us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.430068 (+ 502us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.430294 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":58,"thread_start_us":63,"threads_started":1}
I20260810 14:53:35.431066 6271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.418176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.66:42131)
0810 14:53:35.418678 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.418686 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.419565 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.419831 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.419832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.420274 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.420471 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.420478 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.421461 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.421461 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.422429 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.422432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.422954 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.422955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.423029 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.423499 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.423503 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.426144 (+ 2641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.429233 (+ 3089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.429233 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.429234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.429437 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.429670 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.429671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.429671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.429731 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.430138 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.430138 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.430197 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.430831 (+ 634us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.430954 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":50,"thread_start_us":100,"threads_started":1}
I20260810 14:53:35.431550 6267 heartbeater.cc:347] Connected to a master server at 127.30.74.125:42355
I20260810 14:53:35.431550 6265 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39103
I20260810 14:53:35.431682 6267 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:35.431682 6265 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:35.431922 6265 heartbeater.cc:511] Master 127.30.74.126:39103 requested a full tablet report, sending...
I20260810 14:53:35.431922 6267 heartbeater.cc:511] Master 127.30.74.125:42355 requested a full tablet report, sending...
I20260810 14:53:35.432412 5064 ts_manager.cc:194] Re-registered known tserver with Master: a7835ad107dd430d838123f68e24a29b (127.30.74.66:41633)
I20260810 14:53:35.432474 5708 ts_manager.cc:194] Registered new tserver with Master: a7835ad107dd430d838123f68e24a29b (127.30.74.66:41633)
I20260810 14:53:35.432873 5064 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:42131
I20260810 14:53:35.437335 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:36419
--local_ip_for_outbound_sockets=127.30.74.67
--tserver_master_addrs=127.30.74.126:39103
--webserver_port=37711
--webserver_interface=127.30.74.67
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.67.keytab
--principal=oryx/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.74.126:39103,127.30.74.125:42355
--tserver_master_addrs=127.30.74.126:39103,127.30.74.125:42355,127.30.74.124:34947 with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:53:35.437841 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.418168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34947 (local address 127.30.74.66:50599)
0810 14:53:35.418357 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.418393 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.419545 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.419824 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.419830 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.420186 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.420471 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.420476 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.421203 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.421204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.422033 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.422036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.422514 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.422515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.422652 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.423247 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.423258 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.425054 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.436600 (+ 11546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.436602 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.436603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.436874 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.437102 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.437103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.437104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.437183 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.437452 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.437453 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.437499 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.437588 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.437722 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":105,"threads_started":1}
I20260810 14:53:35.437836 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.418211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:50599 (local address 127.30.74.124:34947)
0810 14:53:35.418260 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.418262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.418603 (+ 341us) server_negotiation.cc:407] Connection header received
0810 14:53:35.419676 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.419677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.419693 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.419738 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.420571 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.421085 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.422134 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.422407 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.422500 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.425173 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.425174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.425174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.425199 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.427257 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.436967 (+ 9710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.436969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.436969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.437039 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.437262 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.437263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.437263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.437378 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.437459 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.437628 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.437750 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:53:35.438417 6266 heartbeater.cc:347] Connected to a master server at 127.30.74.124:34947
I20260810 14:53:35.438484 6266 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:35.438577 6266 heartbeater.cc:511] Master 127.30.74.124:34947 requested a full tablet report, sending...
I20260810 14:53:35.438912 5924 ts_manager.cc:194] Registered new tserver with Master: a7835ad107dd430d838123f68e24a29b (127.30.74.66:41633)
W20260810 14:53:35.524394 6272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:35.524659 6272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:35.524690 6272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:35.525887 6272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:35.525924 6272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:35.525950 6272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:53:35.527505 6272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.67.keytab
--principal=oryx/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:36419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=37711
--tserver_master_addrs=127.30.74.126:39103,127.30.74.125:42355,127.30.74.124:34947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:35.527731 6272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:35.527926 6272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:35.533267 6277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:35.533278 6281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:35.533267 6278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:35.533496 6272 server_base.cc:1034] running on GCE node
I20260810 14:53:35.533872 6272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:35.534166 6272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:35.535328 6272 hybrid_clock.cc:648] HybridClock initialized: now 1786373615535301 us; error 35 us; skew 500 ppm
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373615, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:35.537992 6272 init.cc:377] Logged in from keytab as oryx/127.30.74.67@KRBTEST.COM (short username oryx)
I20260810 14:53:35.538465 6272 webserver.cc:460] Webserver started at http://127.30.74.67:37711/ using document root <none> and password file <none>
I20260810 14:53:35.538705 6272 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:35.538787 6272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:35.540295 6272 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:35.540971 6289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:35.541224 6272 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:35.541332 6272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "9562ca92f305486db348c6c91ed30016"
format_stamp: "Formatted at 2026-08-10 14:53:34 on dist-test-slave-0b58"
server_key: "6b8278dd14270d87856f655ab6b93b18"
server_key_iv: "271ea6f40efce19cbf44b83973df1979"
server_key_version: "encryptionkey@0"
I20260810 14:53:35.541689 6272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:35.554342 6272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:35.556607 6272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:35.556764 6272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:35.557101 6272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:35.557443 6272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:35.557478 6272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:35.557502 6272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:35.557547 6272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:35.563535 6272 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:36419
I20260810 14:53:35.563624 6407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:36419 every 8 connection(s)
I20260810 14:53:35.564008 6272 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373615, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.125@KRBTEST.COM
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373615, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.124@KRBTEST.COM
Aug 10 14:53:35 dist-test-slave-0b58 krb5kdc[5014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373615, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM
I20260810 14:53:35.573586 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 6272
I20260810 14:53:35.573931 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.568583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34947 (local address 127.0.0.1:43298)
0810 14:53:35.568687 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.568697 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.568736 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.569723 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.569724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.569727 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:35.569779 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.569781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.570739 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.570741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.571923 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.571927 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.572837 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.572839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.573204 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.573212 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.573272 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.573334 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260810 14:53:35.574622 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.568813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43298 (local address 127.30.74.124:34947)
0810 14:53:35.569572 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.569574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.569581 (+ 7us) server_negotiation.cc:407] Connection header received
0810 14:53:35.569599 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.569599 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.569620 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.569717 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:35.569893 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.570616 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.572035 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.572761 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.572837 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.572922 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.574493 (+ 1571us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.574549 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":83,"threads_started":1}
I20260810 14:53:35.575333 5935 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 14:53:35.575481 5966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 687bc504a17c410eb594f13b925e956a. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } } }
I20260810 14:53:35.575609 5966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: This master's current role is: LEARNER
I20260810 14:53:35.577666 6414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.564819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34947 (local address 127.30.74.67:56041)
0810 14:53:35.565249 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.565259 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.565710 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.565961 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.565965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.566520 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.566581 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.566583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.567807 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.567809 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.568628 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.568630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.569141 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.569141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.569280 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.570066 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.570072 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.572564 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.574685 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.574693 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.574696 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.575014 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.575660 (+ 646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.575661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.575662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.575819 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.576921 (+ 1102us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.576930 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.577126 (+ 196us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.577237 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.577416 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":3,"spinlock_wait_cycles":768,"thread_start_us":101,"threads_started":1}
I20260810 14:53:35.578039 6124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.564820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:56041 (local address 127.30.74.124:34947)
0810 14:53:35.564877 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.564879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.565320 (+ 441us) server_negotiation.cc:407] Connection header received
0810 14:53:35.565814 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.565816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.565829 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.565863 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.567240 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.567709 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.568729 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.569025 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.569069 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.572665 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.572666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.572666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.572675 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.574611 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.575180 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.575181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.575181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.575264 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.575918 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.575919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.575920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.576071 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.576131 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.577847 (+ 1716us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.577950 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 14:53:35.578599 6409 heartbeater.cc:347] Connected to a master server at 127.30.74.124:34947
I20260810 14:53:35.578981 6409 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:35.579214 6409 heartbeater.cc:511] Master 127.30.74.124:34947 requested a full tablet report, sending...
I20260810 14:53:35.579710 5924 ts_manager.cc:194] Registered new tserver with Master: 9562ca92f305486db348c6c91ed30016 (127.30.74.67:36419)
I20260810 14:53:35.580972 6412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.565131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:42355 (local address 127.30.74.67:58021)
0810 14:53:35.565285 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.565293 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.565709 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.565960 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.565965 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.566371 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.566511 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.566519 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.567299 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.567300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.568505 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.568508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.569053 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.569054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.569280 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.570045 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.570056 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.573051 (+ 2995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.577562 (+ 4511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.577563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.577563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.577982 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.580120 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.580121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.580122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.580196 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.580582 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.580583 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.580642 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.580714 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.580843 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":4,"thread_start_us":59,"threads_started":1}
I20260810 14:53:35.581009 5764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.565252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:58021 (local address 127.30.74.125:42355)
0810 14:53:35.565297 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.565299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.565345 (+ 46us) server_negotiation.cc:407] Connection header received
0810 14:53:35.565814 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.565815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.565829 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.565863 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.566644 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.567147 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.568611 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.568910 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.568953 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.573147 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.573149 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.573150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.573159 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.574935 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.578068 (+ 3133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.578069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.578069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.578133 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.580315 (+ 2182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.580316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.580317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.580484 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.580545 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.580797 (+ 252us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.580889 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:53:35.581581 6410 heartbeater.cc:347] Connected to a master server at 127.30.74.125:42355
I20260810 14:53:35.581661 6410 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:35.581748 6410 heartbeater.cc:511] Master 127.30.74.125:42355 requested a full tablet report, sending...
I20260810 14:53:35.582044 5708 ts_manager.cc:194] Registered new tserver with Master: 9562ca92f305486db348c6c91ed30016 (127.30.74.67:36419)
I20260810 14:53:35.582813 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.565092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:50279 (local address 127.30.74.126:39103)
0810 14:53:35.565167 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.565169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.565180 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:53:35.565803 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.565804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.565817 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.565886 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.567792 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.568319 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.569285 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.569483 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.569566 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.573348 (+ 3782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.573349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.573349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.573357 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.577466 (+ 4109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.581833 (+ 4367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.581835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.581835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.581903 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.582132 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.582133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.582134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.582283 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.582335 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.582617 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.582706 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 14:53:35.583498 6413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.564818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.67:50279)
0810 14:53:35.565012 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.565051 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.565685 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.565960 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.565965 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.566301 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:35.566509 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.566514 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.568419 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.568420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.569191 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.569193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.569636 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.569637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.569726 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.570355 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:35.570361 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:35.573256 (+ 2895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:35.581460 (+ 8204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.581461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.581462 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.581742 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.581987 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:35.581988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:35.581988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:35.582054 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:35.582431 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:35.582432 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:35.582477 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.582907 (+ 430us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.583023 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":23,"thread_start_us":108,"threads_started":1}
I20260810 14:53:35.583976 6408 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39103
I20260810 14:53:35.584012 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.575160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43314 (local address 127.30.74.124:34947)
0810 14:53:35.575193 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.575195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.575204 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:35.575226 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.575226 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.575236 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.575285 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.576082 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.576895 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.577983 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.578290 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.578345 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.579849 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.579850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.579850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.579857 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.582397 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.582831 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.582833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.582833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.582895 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.583116 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.583117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.583117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.583264 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.583371 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.583815 (+ 444us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.583901 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260810 14:53:35.584028 6408 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:35.584105 6408 heartbeater.cc:511] Master 127.30.74.126:39103 requested a full tablet report, sending...
I20260810 14:53:35.584468 5064 ts_manager.cc:194] Re-registered known tserver with Master: 9562ca92f305486db348c6c91ed30016 (127.30.74.67:36419)
I20260810 14:53:35.584936 5064 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:50279
I20260810 14:53:35.585862 6424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.575634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50038 (local address 127.30.74.125:42355)
0810 14:53:35.575764 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.575766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.576338 (+ 572us) server_negotiation.cc:407] Connection header received
0810 14:53:35.576365 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.576366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.576380 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.576440 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.578013 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.578715 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.579717 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.580047 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.580126 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.582629 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.582630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.582631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.582638 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.584613 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.584996 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.584998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.584998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.585063 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.585277 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.585278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.585278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.585403 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.585493 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.585673 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.585766 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260810 14:53:35.586344 6422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.575039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37604 (local address 127.30.74.126:39103)
0810 14:53:35.576004 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.576005 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.576013 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:53:35.576034 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.576035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.576047 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.576118 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.577094 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.578895 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.580101 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.580470 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.580540 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.582324 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.582326 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.582326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.582335 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.584871 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.585291 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.585291 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.585292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.585359 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.585578 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.585579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.585579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.585716 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.586186 (+ 470us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.586221 (+ 35us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.586286 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":925,"thread_start_us":91,"threads_started":1}
I20260810 14:53:35.587970 5064 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:53:35.589522 5064 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:35.590248 5884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: NON_VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: true } } }
I20260810 14:53:35.590364 5884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:35.591107 5884 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a: attempting to promote NON_VOTER 3d4b68df43a641f685798a53d1e6de6f to VOTER
I20260810 14:53:35.591310 5884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } }
I20260810 14:53:35.591750 5935 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 LEARNER]: Refusing update from remote peer 687bc504a17c410eb594f13b925e956a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 14:53:35.591904 5719 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 FOLLOWER]: Refusing update from remote peer 687bc504a17c410eb594f13b925e956a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 14:53:35.591964 6426 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 14:53:35.592206 6426 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 14:53:35.592780 5884 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3d4b68df43a641f685798a53d1e6de6f (127.30.74.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } } }
I20260810 14:53:35.592947 5935 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3d4b68df43a641f685798a53d1e6de6f (127.30.74.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } } }
I20260810 14:53:35.593132 5719 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 3d4b68df43a641f685798a53d1e6de6f (127.30.74.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } } }
I20260810 14:53:35.593752 5884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } } }
I20260810 14:53:35.593840 5884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:35.593904 5884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } } }
I20260810 14:53:35.593957 5884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:35.593958 5966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } } }
I20260810 14:53:35.594002 5884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } } }
I20260810 14:53:35.594030 5966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:53:35.594049 5884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:35.594164 5752 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "687bc504a17c410eb594f13b925e956a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } } }
I20260810 14:53:35.594237 5752 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:53:35.599915 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.595846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:41633 (local address 127.0.0.1:58228)
0810 14:53:35.595983 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.595994 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.596035 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.596583 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.596585 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.596587 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:35.596637 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.596639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.597694 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.597695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.598675 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.598678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.599688 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.599689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.599750 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.599754 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.599783 (+ 29us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.599823 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260810 14:53:35.600359 6436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.596115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58228 (local address 127.30.74.66:41633)
0810 14:53:35.596396 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.596401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.596412 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:53:35.596458 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.596460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.596479 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.596574 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:35.596732 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.597575 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.598770 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.599594 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.600088 (+ 494us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.600178 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.600207 (+ 29us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.600250 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":131,"threads_started":1}
I20260810 14:53:35.600754 6434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.595874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:36419 (local address 127.0.0.1:43344)
0810 14:53:35.596305 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.596316 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.596368 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.596876 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.596877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.596879 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:35.596923 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.596925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.598248 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.598248 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.599379 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.599383 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.600351 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.600352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.600560 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.600567 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.600632 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.600682 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":84,"threads_started":1}
I20260810 14:53:35.600812 6437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.596386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43344 (local address 127.30.74.67:36419)
0810 14:53:35.596648 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.596661 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.596670 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:35.596712 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.596714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.596731 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.596805 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:35.597000 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.598151 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.599481 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.600241 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.600349 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.600431 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.600654 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.600694 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":94,"threads_started":1}
I20260810 14:53:35.600890 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.595947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:35259 (local address 127.0.0.1:59282)
0810 14:53:35.596673 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.596684 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.596788 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.596974 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.596975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.596977 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:35.597021 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.597022 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.598508 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.598509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.599491 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.599494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.600663 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.600664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.600726 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.600730 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.600783 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.600825 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":88,"threads_started":1}
I20260810 14:53:35.601089 6435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.596049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59282 (local address 127.30.74.65:35259)
0810 14:53:35.596234 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.596239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.596772 (+ 533us) server_negotiation.cc:407] Connection header received
0810 14:53:35.596866 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.596869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.596885 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.596964 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:35.597084 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.598425 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.599566 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.600447 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.600518 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.600616 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.600959 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.601004 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":125,"threads_started":1}
I20260810 14:53:35.601058 6195 tablet_service.cc:1467] Processing CreateTablet for tablet ee630eee760f469cb2036dfeb45b99c6 (DEFAULT_TABLE table=test-table [id=f9bc4e3577e64eac94e9f42d8720f611]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:35.601495 6195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee630eee760f469cb2036dfeb45b99c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:35.601804 6340 tablet_service.cc:1467] Processing CreateTablet for tablet ee630eee760f469cb2036dfeb45b99c6 (DEFAULT_TABLE table=test-table [id=f9bc4e3577e64eac94e9f42d8720f611]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:35.602164 6340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee630eee760f469cb2036dfeb45b99c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:35.605434 6438 tablet_bootstrap.cc:492] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b: Bootstrap starting.
I20260810 14:53:35.605996 6050 tablet_service.cc:1467] Processing CreateTablet for tablet ee630eee760f469cb2036dfeb45b99c6 (DEFAULT_TABLE table=test-table [id=f9bc4e3577e64eac94e9f42d8720f611]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:35.606149 6438 tablet_bootstrap.cc:654] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:35.606349 6050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee630eee760f469cb2036dfeb45b99c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:35.606474 6438 log.cc:824] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:35.606876 6441 tablet_bootstrap.cc:492] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016: Bootstrap starting.
I20260810 14:53:35.607406 6438 tablet_bootstrap.cc:492] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b: No bootstrap required, opened a new log
I20260810 14:53:35.607467 6441 tablet_bootstrap.cc:654] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:35.607522 6438 ts_tablet_manager.cc:1397] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:53:35.607827 6441 log.cc:824] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:35.608590 6441 tablet_bootstrap.cc:492] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016: No bootstrap required, opened a new log
I20260810 14:53:35.608685 6441 ts_tablet_manager.cc:1397] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:53:35.609793 6438 raft_consensus.cc:348] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } }
I20260810 14:53:35.609973 6438 raft_consensus.cc:374] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:35.610028 6438 raft_consensus.cc:729] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7835ad107dd430d838123f68e24a29b, State: Initialized, Role: FOLLOWER
I20260810 14:53:35.610114 6445 tablet_bootstrap.cc:492] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31: Bootstrap starting.
I20260810 14:53:35.610178 6438 consensus_queue.cc:260] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } }
I20260810 14:53:35.610487 6438 ts_tablet_manager.cc:1428] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 14:53:35.610502 6265 heartbeater.cc:503] Master 127.30.74.126:39103 was elected leader, sending a full tablet report...
I20260810 14:53:35.610457 6441 raft_consensus.cc:348] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } }
I20260810 14:53:35.610620 6441 raft_consensus.cc:374] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:35.610661 6441 raft_consensus.cc:729] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9562ca92f305486db348c6c91ed30016, State: Initialized, Role: FOLLOWER
I20260810 14:53:35.610818 6445 tablet_bootstrap.cc:654] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:35.610802 6441 consensus_queue.cc:260] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } }
I20260810 14:53:35.611073 6441 ts_tablet_manager.cc:1428] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:53:35.611195 6408 heartbeater.cc:503] Master 127.30.74.126:39103 was elected leader, sending a full tablet report...
I20260810 14:53:35.611153 6445 log.cc:824] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:35.613315 6445 tablet_bootstrap.cc:492] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31: No bootstrap required, opened a new log
I20260810 14:53:35.613409 6445 ts_tablet_manager.cc:1397] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 14:53:35.614789 6445 raft_consensus.cc:348] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } }
I20260810 14:53:35.614945 6445 raft_consensus.cc:374] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:35.614992 6445 raft_consensus.cc:729] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd160ee324f54d4d9d942d1ed6215b31, State: Initialized, Role: FOLLOWER
I20260810 14:53:35.615126 6445 consensus_queue.cc:260] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } }
I20260810 14:53:35.615389 6445 ts_tablet_manager.cc:1428] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:53:35.615448 6120 heartbeater.cc:503] Master 127.30.74.126:39103 was elected leader, sending a full tablet report...
I20260810 14:53:35.632354 6446 raft_consensus.cc:482] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:35.632560 6446 raft_consensus.cc:504] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } }
I20260810 14:53:35.632987 6446 leader_election.cc:288] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9562ca92f305486db348c6c91ed30016 (127.30.74.67:36419), dd160ee324f54d4d9d942d1ed6215b31 (127.30.74.65:35259)
I20260810 14:53:35.636935 6437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.633264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:33089 (local address 127.30.74.67:36419)
0810 14:53:35.633340 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.633343 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.633467 (+ 124us) server_negotiation.cc:407] Connection header received
0810 14:53:35.633504 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.633505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.633514 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.633563 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:35.633677 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.634401 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.635965 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.636557 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.636637 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.636645 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.636820 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.636860 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 14:53:35.636935 6435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.633234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:54255 (local address 127.30.74.65:35259)
0810 14:53:35.633276 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.633278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.633290 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:53:35.633361 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.633362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.633372 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.633455 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:35.633640 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.634404 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.635964 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.636557 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.636637 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.636645 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.636820 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.636860 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:53:35.636967 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.633175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:36419 (local address 127.30.74.66:33089)
0810 14:53:35.633399 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.633410 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.633458 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.633567 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.633568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.633569 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:35.633624 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.633625 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.634516 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.634517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.635862 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.635866 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.636645 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.636647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.636733 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.636737 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.636801 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.636849 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":253}
I20260810 14:53:35.636981 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.633173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:35259 (local address 127.30.74.66:54255)
0810 14:53:35.633233 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:35.633248 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:35.633292 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:35.633444 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:35.633445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:35.633447 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:35.633544 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.633546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.634515 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.634517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.635863 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:35.635866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.636659 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.636660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:35.636733 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.636737 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:35.636801 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 14:53:35.636849 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":67}
I20260810 14:53:35.637607 6361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee630eee760f469cb2036dfeb45b99c6" candidate_uuid: "a7835ad107dd430d838123f68e24a29b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9562ca92f305486db348c6c91ed30016" is_pre_election: true
I20260810 14:53:35.637748 6361 raft_consensus.cc:2454] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7835ad107dd430d838123f68e24a29b in term 0.
I20260810 14:53:35.637759 6071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee630eee760f469cb2036dfeb45b99c6" candidate_uuid: "a7835ad107dd430d838123f68e24a29b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd160ee324f54d4d9d942d1ed6215b31" is_pre_election: true
I20260810 14:53:35.637912 6071 raft_consensus.cc:2454] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7835ad107dd430d838123f68e24a29b in term 0.
I20260810 14:53:35.637991 6146 leader_election.cc:302] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9562ca92f305486db348c6c91ed30016, a7835ad107dd430d838123f68e24a29b; no voters:
I20260810 14:53:35.638128 6446 raft_consensus.cc:2781] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:35.638185 6446 raft_consensus.cc:482] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:35.638238 6446 raft_consensus.cc:3037] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:35.638969 6446 raft_consensus.cc:504] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } }
I20260810 14:53:35.639163 6446 leader_election.cc:288] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [CANDIDATE]: Term 1 election: Requested vote from peers 9562ca92f305486db348c6c91ed30016 (127.30.74.67:36419), dd160ee324f54d4d9d942d1ed6215b31 (127.30.74.65:35259)
I20260810 14:53:35.639318 6071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee630eee760f469cb2036dfeb45b99c6" candidate_uuid: "a7835ad107dd430d838123f68e24a29b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd160ee324f54d4d9d942d1ed6215b31"
I20260810 14:53:35.639318 6361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee630eee760f469cb2036dfeb45b99c6" candidate_uuid: "a7835ad107dd430d838123f68e24a29b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9562ca92f305486db348c6c91ed30016"
I20260810 14:53:35.639415 6071 raft_consensus.cc:3037] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:35.639425 6361 raft_consensus.cc:3037] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:35.640108 6071 raft_consensus.cc:2454] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7835ad107dd430d838123f68e24a29b in term 1.
I20260810 14:53:35.640216 6361 raft_consensus.cc:2454] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7835ad107dd430d838123f68e24a29b in term 1.
I20260810 14:53:35.640413 6147 leader_election.cc:302] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7835ad107dd430d838123f68e24a29b, dd160ee324f54d4d9d942d1ed6215b31; no voters:
I20260810 14:53:35.640559 6446 raft_consensus.cc:2781] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:35.640758 6446 raft_consensus.cc:686] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [term 1 LEADER]: Becoming Leader. State: Replica: a7835ad107dd430d838123f68e24a29b, State: Running, Role: LEADER
I20260810 14:53:35.640872 6446 consensus_queue.cc:237] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } }
I20260810 14:53:35.641722 5065 catalog_manager.cc:5495] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b reported cstate change: term changed from 0 to 1, leader changed from <none> to a7835ad107dd430d838123f68e24a29b (127.30.74.66). New cstate: current_term: 1 leader_uuid: "a7835ad107dd430d838123f68e24a29b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } health_report { overall_health: UNKNOWN } } }
I20260810 14:53:35.659230 6436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.656801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58230 (local address 127.30.74.66:41633)
0810 14:53:35.656852 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.656854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.656861 (+ 7us) server_negotiation.cc:407] Connection header received
0810 14:53:35.656897 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.656897 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.656914 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.656981 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:35.657153 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.657711 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.658462 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.658648 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.658704 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.658892 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:35.659012 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:35.659121 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.659134 (+ 13us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.659156 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:53:35.661753 6361 raft_consensus.cc:1261] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016 [term 1 FOLLOWER]: Refusing update from remote peer a7835ad107dd430d838123f68e24a29b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:35.661819 6071 raft_consensus.cc:1261] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31 [term 1 FOLLOWER]: Refusing update from remote peer a7835ad107dd430d838123f68e24a29b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:35.662178 6446 consensus_queue.cc:1035] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:35.662362 6451 consensus_queue.cc:1035] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:35.665413 6455 mvcc.cc:205] Tried to move back new op lower bound from 7316986329746993152 to 7316986329665703936. Current Snapshot: MvccSnapshot[applied={T|T < 7316986329746993152}]
I20260810 14:53:35.669375 5065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:35.669466 5065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:35.670840 5065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 687bc504a17c410eb594f13b925e956a: Sending DeleteTablet for 3 replicas of tablet ee630eee760f469cb2036dfeb45b99c6
I20260810 14:53:35.671195 6195 tablet_service.cc:1514] Processing DeleteTablet for tablet ee630eee760f469cb2036dfeb45b99c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:35 UTC) from {username='oryx', principal='oryx/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:58228
I20260810 14:53:35.671218 6050 tablet_service.cc:1514] Processing DeleteTablet for tablet ee630eee760f469cb2036dfeb45b99c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:35 UTC) from {username='oryx', principal='oryx/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:59282
I20260810 14:53:35.671452 6340 tablet_service.cc:1514] Processing DeleteTablet for tablet ee630eee760f469cb2036dfeb45b99c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:35 UTC) from {username='oryx', principal='oryx/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:43344
I20260810 14:53:35.671708 6468 tablet_replica.cc:330] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016: stopping tablet replica
I20260810 14:53:35.671839 6468 raft_consensus.cc:2229] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:35.671978 6468 raft_consensus.cc:2258] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:35.672541 6468 ts_tablet_manager.cc:1905] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:35.673571 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 5027
I20260810 14:53:35.675367 6468 ts_tablet_manager.cc:1918] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:35.675465 6468 log.cc:1196] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/ee630eee760f469cb2036dfeb45b99c6
I20260810 14:53:35.675781 6468 ts_tablet_manager.cc:1939] T ee630eee760f469cb2036dfeb45b99c6 P 9562ca92f305486db348c6c91ed30016: Deleting consensus metadata
I20260810 14:53:35.676164 6466 tablet_replica.cc:330] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b: stopping tablet replica
I20260810 14:53:35.672003 6467 tablet_replica.cc:330] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31: stopping tablet replica
I20260810 14:53:35.678862 6467 raft_consensus.cc:2229] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:35.679009 6467 raft_consensus.cc:2258] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:35.679189 6466 raft_consensus.cc:2229] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:53:35.679340 6466 raft_consensus.cc:2258] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:35.679937 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.672313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43324 (local address 127.30.74.124:34947)
0810 14:53:35.672357 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.672358 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.672367 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:35.672388 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.672389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.672400 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.672475 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.673307 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.674014 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.675038 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.675350 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.675403 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.677027 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.677028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.677029 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.677037 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.678552 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.678911 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.678913 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.678913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.678977 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.679191 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.679192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.679192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.679324 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.679442 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.679756 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.679847 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:53:35.679979 6466 ts_tablet_manager.cc:1905] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:35.681427 6466 ts_tablet_manager.cc:1918] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:35.681524 6466 log.cc:1196] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/ee630eee760f469cb2036dfeb45b99c6
I20260810 14:53:35.681780 6466 ts_tablet_manager.cc:1939] T ee630eee760f469cb2036dfeb45b99c6 P a7835ad107dd430d838123f68e24a29b: Deleting consensus metadata
I20260810 14:53:35.684650 6467 ts_tablet_manager.cc:1905] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 14:53:35.684975 6290 connection.cc:535] server connection from 127.0.0.1:43344 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 14:53:35.685341 6146 connection.cc:535] server connection from 127.0.0.1:58228 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 14:53:35.686324 6467 ts_tablet_manager.cc:1918] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:35.686408 6467 log.cc:1196] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-0/wal/wals/ee630eee760f469cb2036dfeb45b99c6
I20260810 14:53:35.686704 6467 ts_tablet_manager.cc:1939] T ee630eee760f469cb2036dfeb45b99c6 P dd160ee324f54d4d9d942d1ed6215b31: Deleting consensus metadata
W20260810 14:53:35.687081 5998 connection.cc:441] server connection from 127.0.0.1:59282 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59282 (request call id 1) could send its response
I20260810 14:53:35.695287 6415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.687745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43332 (local address 127.30.74.124:34947)
0810 14:53:35.687793 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.687795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.687934 (+ 139us) server_negotiation.cc:407] Connection header received
0810 14:53:35.687980 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.687981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.687991 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.688035 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.688999 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.689481 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.690181 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.690396 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.690473 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.692206 (+ 1733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.692207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.692207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.692216 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.694004 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.694488 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.694490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.694490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.694552 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.694739 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.694740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.694740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.694859 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.694961 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.695077 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.695154 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 14:53:35.695466 6424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:35.687576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50054 (local address 127.30.74.125:42355)
0810 14:53:35.687623 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:35.687625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:35.687634 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:35.687660 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:35.687661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:35.687675 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:35.687739 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:35.688553 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.689352 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.690478 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:35.690699 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:35.690752 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:35.692403 (+ 1651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:35.692404 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:35.692405 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:35.692413 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:35.694130 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.694525 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.694526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.694526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.694590 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:35.694803 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:35.694804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:35.694804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:35.694956 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:35.695028 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:35.695274 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 14:53:35.695370 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:53:36.667757 6476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:36.667475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.67:48617)
0810 14:53:36.667604 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:36.667694 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260810 14:53:36.668094 6477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:36.667750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.65:60993)
0810 14:53:36.667974 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:36.668050 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":56,"threads_started":1}
I20260810 14:53:36.668243 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:36.667874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.66:38419)
0810 14:53:36.668037 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:36.668120 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":57,"threads_started":1}
W20260810 14:53:36.668341 6120 heartbeater.cc:644] Failed to heartbeat to 127.30.74.126:39103 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
W20260810 14:53:36.668527 6408 heartbeater.cc:644] Failed to heartbeat to 127.30.74.126:39103 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
W20260810 14:53:36.668704 6265 heartbeater.cc:644] Failed to heartbeat to 127.30.74.126:39103 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
I20260810 14:53:37.171988 6479 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 687bc504a17c410eb594f13b925e956a)
I20260810 14:53:37.172134 6479 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } }
I20260810 14:53:37.172452 6479 leader_election.cc:288] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 687bc504a17c410eb594f13b925e956a (127.30.74.126:39103), fcc444f3a3e44c90b4aeb42775a3a4f3 (127.30.74.125:42355)
I20260810 14:53:37.172943 6480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:37.172685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:38110)
0810 14:53:37.172810 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:37.172871 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":81,"threads_started":1}
W20260810 14:53:37.173106 5908 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
I20260810 14:53:37.173439 6483 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 687bc504a17c410eb594f13b925e956a)
I20260810 14:53:37.173607 6480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:37.173455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:38124)
0810 14:53:37.173545 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:37.173582 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 14:53:37.173569 6483 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } }
W20260810 14:53:37.173673 5908 leader_election.cc:334] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 687bc504a17c410eb594f13b925e956a (127.30.74.126:39103): Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
I20260810 14:53:37.173880 6483 leader_election.cc:288] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 687bc504a17c410eb594f13b925e956a (127.30.74.126:39103), 3d4b68df43a641f685798a53d1e6de6f (127.30.74.124:34947)
I20260810 14:53:37.174275 6489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:37.173947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:38130)
0810 14:53:37.174118 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:37.174229 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":58,"threads_started":1}
W20260810 14:53:37.174414 5693 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
I20260810 14:53:37.174903 6489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:37.174759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:38132)
0810 14:53:37.174821 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:37.174873 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 14:53:37.175060 5693 leader_election.cc:334] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 687bc504a17c410eb594f13b925e956a (127.30.74.126:39103): Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
I20260810 14:53:37.176820 6481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:37.172885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:42355 (local address 127.0.0.1:50066)
0810 14:53:37.172974 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:37.172984 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:37.173290 (+ 306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:37.173508 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:37.173509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:37.173513 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:37.173577 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:37.173580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:37.174685 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:37.174686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:37.175943 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:37.175946 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:37.176572 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:37.176573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:37.176641 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:37.176645 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:37.176696 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 14:53:37.176768 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":30,"thread_start_us":38,"threads_started":1}
I20260810 14:53:37.176841 6482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:37.172998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50066 (local address 127.30.74.125:42355)
0810 14:53:37.173194 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:37.173197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:37.173210 (+ 13us) server_negotiation.cc:407] Connection header received
0810 14:53:37.173402 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:37.173404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:37.173420 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:37.173483 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:37.173778 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:37.174600 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:37.176036 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:37.176485 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:37.176569 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:37.176583 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:37.176720 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 14:53:37.176766 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":96,"threads_started":1}
I20260810 14:53:37.177335 5719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d4b68df43a641f685798a53d1e6de6f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" is_pre_election: true
I20260810 14:53:37.177476 5719 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d4b68df43a641f685798a53d1e6de6f in term 1.
I20260810 14:53:37.177486 6487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:37.174002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34947 (local address 127.0.0.1:43344)
0810 14:53:37.174144 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:37.174156 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:37.174254 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:37.174453 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:37.174454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:37.174456 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:37.174503 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:37.174505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:37.175394 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:37.175395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:37.176493 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:37.176496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:37.177220 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:37.177221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:37.177290 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:37.177293 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:53:37.177373 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 14:53:37.177416 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":55,"threads_started":1}
I20260810 14:53:37.177500 6490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:37.174115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43344 (local address 127.30.74.124:34947)
0810 14:53:37.174247 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:37.174249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:37.174257 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:53:37.174325 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:37.174326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:37.174338 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:37.174401 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:37.174574 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:37.175303 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:37.176595 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:37.177113 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:37.177214 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:37.177225 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:37.177381 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 14:53:37.177425 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260810 14:53:37.177790 5935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3d4b68df43a641f685798a53d1e6de6f" is_pre_election: true
I20260810 14:53:37.177865 5908 leader_election.cc:302] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d4b68df43a641f685798a53d1e6de6f, fcc444f3a3e44c90b4aeb42775a3a4f3; no voters: 687bc504a17c410eb594f13b925e956a
I20260810 14:53:37.177906 5935 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcc444f3a3e44c90b4aeb42775a3a4f3 in term 1.
I20260810 14:53:37.177969 6479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 14:53:37.178004 6479 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 687bc504a17c410eb594f13b925e956a)
I20260810 14:53:37.178028 6479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:53:37.178117 5695 leader_election.cc:302] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d4b68df43a641f685798a53d1e6de6f, fcc444f3a3e44c90b4aeb42775a3a4f3; no voters: 687bc504a17c410eb594f13b925e956a
I20260810 14:53:37.178232 6483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 14:53:37.178268 6483 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 687bc504a17c410eb594f13b925e956a)
I20260810 14:53:37.178282 6483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:53:37.178930 6479 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } }
I20260810 14:53:37.178961 6483 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } }
I20260810 14:53:37.179082 6479 leader_election.cc:288] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [CANDIDATE]: Term 2 election: Requested vote from peers 687bc504a17c410eb594f13b925e956a (127.30.74.126:39103), fcc444f3a3e44c90b4aeb42775a3a4f3 (127.30.74.125:42355)
I20260810 14:53:37.179122 6483 leader_election.cc:288] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [CANDIDATE]: Term 2 election: Requested vote from peers 687bc504a17c410eb594f13b925e956a (127.30.74.126:39103), 3d4b68df43a641f685798a53d1e6de6f (127.30.74.124:34947)
I20260810 14:53:37.179291 6481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:37.179196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:38148)
0810 14:53:37.179233 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:37.179263 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260810 14:53:37.179324 5935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3d4b68df43a641f685798a53d1e6de6f"
I20260810 14:53:37.179363 5719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d4b68df43a641f685798a53d1e6de6f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3"
I20260810 14:53:37.179442 5935 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fcc444f3a3e44c90b4aeb42775a3a4f3 in current term 2: Already voted for candidate 3d4b68df43a641f685798a53d1e6de6f in this term.
I20260810 14:53:37.179445 5719 raft_consensus.cc:2379] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d4b68df43a641f685798a53d1e6de6f in current term 2: Already voted for candidate fcc444f3a3e44c90b4aeb42775a3a4f3 in this term.
I20260810 14:53:37.179708 6487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:37.179280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:38164)
0810 14:53:37.179633 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:37.179662 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339}
I20260810 14:53:37.179833 6481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:37.179745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:38180)
0810 14:53:37.179781 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:37.179805 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260810 14:53:37.179900 5908 leader_election.cc:334] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 687bc504a17c410eb594f13b925e956a (127.30.74.126:39103): Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
I20260810 14:53:37.179968 5908 leader_election.cc:302] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d4b68df43a641f685798a53d1e6de6f; no voters: 687bc504a17c410eb594f13b925e956a, fcc444f3a3e44c90b4aeb42775a3a4f3
I20260810 14:53:37.180178 6479 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 14:53:37.180220 6487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:37.180128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:38196)
0810 14:53:37.180157 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:37.180184 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260810 14:53:37.180289 5693 leader_election.cc:334] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 687bc504a17c410eb594f13b925e956a (127.30.74.126:39103): Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
I20260810 14:53:37.180342 5693 leader_election.cc:302] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcc444f3a3e44c90b4aeb42775a3a4f3; no voters: 3d4b68df43a641f685798a53d1e6de6f, 687bc504a17c410eb594f13b925e956a
I20260810 14:53:37.180471 6483 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 14:53:37.668998 6492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:37.668746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.65:60709)
0810 14:53:37.668896 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:37.668939 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260810 14:53:37.669538 6493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:37.669071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.66:47819)
0810 14:53:37.669438 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:37.669479 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":57,"threads_started":1}
I20260810 14:53:37.670141 6494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:37.669952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.67:44169)
0810 14:53:37.670071 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:37.670106 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
I20260810 14:53:38.669889 6495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.669411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.65:35923)
0810 14:53:38.669708 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.669824 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":101,"threads_started":1}
I20260810 14:53:38.670181 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.669895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.66:47195)
0810 14:53:38.670091 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.670129 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":57,"threads_started":1}
I20260810 14:53:38.670795 6497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.670546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.67:45593)
0810 14:53:38.670706 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.670748 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":92,"threads_started":1}
I20260810 14:53:38.685387 6498 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:38.685515 6498 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } }
I20260810 14:53:38.685771 6498 leader_election.cc:288] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 687bc504a17c410eb594f13b925e956a (127.30.74.126:39103), 3d4b68df43a641f685798a53d1e6de6f (127.30.74.124:34947)
I20260810 14:53:38.686044 5935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3d4b68df43a641f685798a53d1e6de6f" is_pre_election: true
I20260810 14:53:38.686143 6499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.685918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:38400)
0810 14:53:38.686030 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.686087 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":62,"threads_started":1}
I20260810 14:53:38.686173 5935 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcc444f3a3e44c90b4aeb42775a3a4f3 in term 2.
I20260810 14:53:38.686461 5695 leader_election.cc:302] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d4b68df43a641f685798a53d1e6de6f, fcc444f3a3e44c90b4aeb42775a3a4f3; no voters:
I20260810 14:53:38.686589 6498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260810 14:53:38.686638 6498 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:38.686659 6498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 2 FOLLOWER]: Advancing to term 3
I20260810 14:53:38.686738 6499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.686622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:38410)
0810 14:53:38.686683 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.686712 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 14:53:38.686833 5693 leader_election.cc:334] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 687bc504a17c410eb594f13b925e956a (127.30.74.126:39103): Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
I20260810 14:53:38.687520 6498 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } }
I20260810 14:53:38.687672 6498 leader_election.cc:288] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [CANDIDATE]: Term 3 election: Requested vote from peers 687bc504a17c410eb594f13b925e956a (127.30.74.126:39103), 3d4b68df43a641f685798a53d1e6de6f (127.30.74.124:34947)
I20260810 14:53:38.687844 6499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.687759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:38426)
0810 14:53:38.687794 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.687820 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260810 14:53:38.687938 5935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3d4b68df43a641f685798a53d1e6de6f"
I20260810 14:53:38.688031 5935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 2 FOLLOWER]: Advancing to term 3
I20260810 14:53:38.688243 6499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.688130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:38442)
0810 14:53:38.688184 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.688215 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 14:53:38.688342 5693 leader_election.cc:334] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 687bc504a17c410eb594f13b925e956a (127.30.74.126:39103): Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
I20260810 14:53:38.688827 5935 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcc444f3a3e44c90b4aeb42775a3a4f3 in term 3.
I20260810 14:53:38.689031 5695 leader_election.cc:302] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d4b68df43a641f685798a53d1e6de6f, fcc444f3a3e44c90b4aeb42775a3a4f3; no voters: 687bc504a17c410eb594f13b925e956a
I20260810 14:53:38.689126 6498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 14:53:38.689236 6498 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [term 3 LEADER]: Becoming Leader. State: Replica: fcc444f3a3e44c90b4aeb42775a3a4f3, State: Running, Role: LEADER
I20260810 14:53:38.689319 6498 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } }
I20260810 14:53:38.689679 6501 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcc444f3a3e44c90b4aeb42775a3a4f3. Latest consensus state: current_term: 3 leader_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } } }
I20260810 14:53:38.689800 6501 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:38.689992 6503 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:38.690506 6503 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:38.690613 6503 catalog_manager.cc:1259] Loaded cluster ID: 40a0197fd68d4fcd8362e3445970b91c
I20260810 14:53:38.690645 6503 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:53:38.690838 6503 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:38.690938 6503 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: Loaded TSK: 0
I20260810 14:53:38.691079 6503 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:53:38.711527 5708 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:53:38.713332 5708 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:38.714514 6499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.714373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:38472)
0810 14:53:38.714440 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.714478 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 14:53:38.714545 5935 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [term 3 FOLLOWER]: Refusing update from remote peer fcc444f3a3e44c90b4aeb42775a3a4f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260810 14:53:38.714866 6499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.714763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.0.0.1:38476)
0810 14:53:38.714813 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.714844 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 14:53:38.714891 6501 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260810 14:53:38.715059 5693 consensus_peers.cc:487] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 -> Peer 687bc504a17c410eb594f13b925e956a (127.30.74.126:39103): Couldn't send request to peer 687bc504a17c410eb594f13b925e956a. Status: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 14:53:38.715587 6505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcc444f3a3e44c90b4aeb42775a3a4f3. Latest consensus state: current_term: 3 leader_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } } }
I20260810 14:53:38.715710 6505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:53:38.716141 6498 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } } }
I20260810 14:53:38.716315 6498 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:38.716876 6505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "687bc504a17c410eb594f13b925e956a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39103 } } peers { permanent_uuid: "fcc444f3a3e44c90b4aeb42775a3a4f3" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 42355 } attrs { promote: false } } peers { permanent_uuid: "3d4b68df43a641f685798a53d1e6de6f" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34947 } attrs { promote: false } } }
I20260810 14:53:38.717000 6505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d4b68df43a641f685798a53d1e6de6f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:53:38.717088 6506 mvcc.cc:205] Tried to move back new op lower bound from 7316986342252138496 to 7316986342152073216. Current Snapshot: MvccSnapshot[applied={T|T < 7316986342252138496}]
I20260810 14:53:38.721526 6511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.718210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:36419 (local address 127.0.0.1:43354)
0810 14:53:38.718516 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.718528 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:38.718589 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:38.718793 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:38.718794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:38.718796 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:38.718828 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:38.718829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.719470 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.719471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:38.720620 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:38.720623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.721289 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.721290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:38.721352 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:38.721357 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:53:38.721415 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 14:53:38.721454 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":94,"threads_started":1}
I20260810 14:53:38.721812 6499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.718138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:35259 (local address 127.0.0.1:59298)
0810 14:53:38.718169 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.718182 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:38.718238 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:38.718722 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:38.718724 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:38.718726 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:38.718785 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:38.718787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.719741 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.719742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:38.720830 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:38.720832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.721597 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.721598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:38.721675 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:38.721678 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:53:38.721707 (+ 29us) client_negotiation.cc:240] Negotiation successful
0810 14:53:38.721748 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":36}
I20260810 14:53:38.721808 6513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.718410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43354 (local address 127.30.74.67:36419)
0810 14:53:38.718516 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:38.718518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:38.718575 (+ 57us) server_negotiation.cc:407] Connection header received
0810 14:53:38.718644 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:38.718645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:38.718658 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:38.718749 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:38.718899 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.719384 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.720692 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.721200 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.721275 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:38.721288 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:38.721433 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 14:53:38.721473 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":55,"threads_started":1}
I20260810 14:53:38.722034 6340 tablet_service.cc:1467] Processing CreateTablet for tablet 91ade0402f7848bd9f87aac726082454 (DEFAULT_TABLE table=test-table [id=7952bda61ee24f8cbadc4098d67768cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:38.722117 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.718333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59298 (local address 127.30.74.65:35259)
0810 14:53:38.718577 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:38.718579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:38.718588 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:38.718615 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:38.718616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:38.718628 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:38.718719 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:38.718867 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.719647 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.720906 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.721498 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.721990 (+ 492us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:38.722005 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:38.722023 (+ 18us) server_negotiation.cc:299] Negotiation successful
0810 14:53:38.722067 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":72,"threads_started":1}
I20260810 14:53:38.722199 6340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91ade0402f7848bd9f87aac726082454. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:38.722338 6050 tablet_service.cc:1467] Processing CreateTablet for tablet 91ade0402f7848bd9f87aac726082454 (DEFAULT_TABLE table=test-table [id=7952bda61ee24f8cbadc4098d67768cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:38.722460 6050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91ade0402f7848bd9f87aac726082454. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:38.723805 6515 tablet_bootstrap.cc:492] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016: Bootstrap starting.
I20260810 14:53:38.723837 6516 tablet_bootstrap.cc:492] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31: Bootstrap starting.
I20260810 14:53:38.724331 6515 tablet_bootstrap.cc:654] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:38.724331 6516 tablet_bootstrap.cc:654] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:38.724742 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.718460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58232 (local address 127.30.74.66:41633)
0810 14:53:38.718648 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:38.718650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:38.718790 (+ 140us) server_negotiation.cc:407] Connection header received
0810 14:53:38.718821 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:38.718822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:38.718834 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:38.718906 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:38.719068 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.719773 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.721732 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.724241 (+ 2509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.724365 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:38.724380 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:38.724617 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 14:53:38.724670 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":54,"threads_started":1}
I20260810 14:53:38.724761 6510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.718210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:41633 (local address 127.0.0.1:58232)
0810 14:53:38.718677 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.718686 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:38.718731 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:38.718951 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:38.718951 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:38.718953 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:38.718995 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:38.718997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.719853 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.719854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:38.721645 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:38.721648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.724370 (+ 2722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.724371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:38.724488 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:38.724492 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:38.724609 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 14:53:38.724665 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":3,"thread_start_us":90,"threads_started":1}
I20260810 14:53:38.725085 6195 tablet_service.cc:1467] Processing CreateTablet for tablet 91ade0402f7848bd9f87aac726082454 (DEFAULT_TABLE table=test-table [id=7952bda61ee24f8cbadc4098d67768cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:38.725214 6195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91ade0402f7848bd9f87aac726082454. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:38.725219 6516 tablet_bootstrap.cc:492] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31: No bootstrap required, opened a new log
I20260810 14:53:38.725276 6516 ts_tablet_manager.cc:1397] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:38.725404 6516 raft_consensus.cc:348] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } }
I20260810 14:53:38.725484 6516 raft_consensus.cc:374] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:38.725512 6516 raft_consensus.cc:729] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd160ee324f54d4d9d942d1ed6215b31, State: Initialized, Role: FOLLOWER
I20260810 14:53:38.725566 6516 consensus_queue.cc:260] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } }
I20260810 14:53:38.725769 6516 ts_tablet_manager.cc:1428] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 14:53:38.725988 6515 tablet_bootstrap.cc:492] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016: No bootstrap required, opened a new log
I20260810 14:53:38.726058 6515 ts_tablet_manager.cc:1397] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 14:53:38.726197 6515 raft_consensus.cc:348] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } }
I20260810 14:53:38.726315 6515 raft_consensus.cc:374] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:38.726351 6515 raft_consensus.cc:729] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9562ca92f305486db348c6c91ed30016, State: Initialized, Role: FOLLOWER
I20260810 14:53:38.726426 6515 consensus_queue.cc:260] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } }
I20260810 14:53:38.726564 6520 tablet_bootstrap.cc:492] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b: Bootstrap starting.
I20260810 14:53:38.726624 6515 ts_tablet_manager.cc:1428] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:38.726888 6495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.726777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.65:48605)
0810 14:53:38.726822 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.726857 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260810 14:53:38.726933 6497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.726839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.67:41251)
0810 14:53:38.726883 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.726906 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 14:53:38.726985 6520 tablet_bootstrap.cc:654] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:38.727561 6122 heartbeater.cc:503] Master 127.30.74.125:42355 was elected leader, sending a full tablet report...
I20260810 14:53:38.727617 6410 heartbeater.cc:503] Master 127.30.74.125:42355 was elected leader, sending a full tablet report...
I20260810 14:53:38.727888 6495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.727662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.65:58765)
0810 14:53:38.727825 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.727856 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260810 14:53:38.727913 6497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.727822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.67:41997)
0810 14:53:38.727858 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.727887 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260810 14:53:38.728163 6520 tablet_bootstrap.cc:492] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b: No bootstrap required, opened a new log
I20260810 14:53:38.728216 6520 ts_tablet_manager.cc:1397] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:53:38.728348 6520 raft_consensus.cc:348] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } }
I20260810 14:53:38.728402 6520 raft_consensus.cc:374] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:38.728415 6520 raft_consensus.cc:729] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7835ad107dd430d838123f68e24a29b, State: Initialized, Role: FOLLOWER
I20260810 14:53:38.728473 6520 consensus_queue.cc:260] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } }
I20260810 14:53:38.728641 6520 ts_tablet_manager.cc:1428] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:38.728919 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.728800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.66:45869)
0810 14:53:38.728857 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.728891 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260810 14:53:38.729185 6267 heartbeater.cc:503] Master 127.30.74.125:42355 was elected leader, sending a full tablet report...
W20260810 14:53:38.770031 6123 tablet.cc:2369] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:53:38.816841 6411 tablet.cc:2369] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:53:38.861127 6521 raft_consensus.cc:482] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:38.861246 6521 raft_consensus.cc:504] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } }
I20260810 14:53:38.861601 6521 leader_election.cc:288] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd160ee324f54d4d9d942d1ed6215b31 (127.30.74.65:35259), a7835ad107dd430d838123f68e24a29b (127.30.74.66:41633)
I20260810 14:53:38.864620 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.862041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:55257 (local address 127.30.74.66:41633)
0810 14:53:38.862086 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:38.862088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:38.862096 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:53:38.862123 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:38.862123 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:38.862134 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:38.862235 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:38.862416 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.862881 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.863855 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.864299 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.864373 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:38.864384 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:38.864521 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 14:53:38.864561 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 14:53:38.864611 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.861853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:57555 (local address 127.30.74.65:35259)
0810 14:53:38.861897 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:38.861899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:38.861909 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:38.862054 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:38.862055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:38.862066 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:38.862120 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:38.862331 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.862850 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.863852 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.864209 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.864330 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:38.864343 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:38.864486 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 14:53:38.864531 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 14:53:38.864616 6497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.861766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:35259 (local address 127.30.74.67:57555)
0810 14:53:38.861830 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.861844 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:38.861906 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:38.862148 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:38.862149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:38.862151 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:38.862234 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:38.862237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.862941 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.862942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:38.863750 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:38.863752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.864321 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.864323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:38.864404 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:38.864409 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:53:38.864463 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 14:53:38.864527 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 14:53:38.864648 6524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.861799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:41633 (local address 127.30.74.67:55257)
0810 14:53:38.861999 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.862011 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:38.862065 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:38.862260 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:38.862261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:38.862264 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:38.862334 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:38.862336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.863002 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.863003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:38.863746 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:38.863748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.864374 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.864375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:38.864427 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:38.864430 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:53:38.864505 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 14:53:38.864544 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":1,"thread_start_us":114,"threads_started":1}
I20260810 14:53:38.864924 6071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91ade0402f7848bd9f87aac726082454" candidate_uuid: "9562ca92f305486db348c6c91ed30016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd160ee324f54d4d9d942d1ed6215b31" is_pre_election: true
I20260810 14:53:38.864933 6216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91ade0402f7848bd9f87aac726082454" candidate_uuid: "9562ca92f305486db348c6c91ed30016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7835ad107dd430d838123f68e24a29b" is_pre_election: true
I20260810 14:53:38.865033 6071 raft_consensus.cc:2454] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9562ca92f305486db348c6c91ed30016 in term 0.
I20260810 14:53:38.865053 6216 raft_consensus.cc:2454] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9562ca92f305486db348c6c91ed30016 in term 0.
I20260810 14:53:38.865255 6290 leader_election.cc:302] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9562ca92f305486db348c6c91ed30016, a7835ad107dd430d838123f68e24a29b; no voters:
I20260810 14:53:38.865396 6521 raft_consensus.cc:2781] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:38.865443 6521 raft_consensus.cc:482] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:38.865465 6521 raft_consensus.cc:3037] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:38.866216 6521 raft_consensus.cc:504] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } }
I20260810 14:53:38.866377 6521 leader_election.cc:288] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [CANDIDATE]: Term 1 election: Requested vote from peers dd160ee324f54d4d9d942d1ed6215b31 (127.30.74.65:35259), a7835ad107dd430d838123f68e24a29b (127.30.74.66:41633)
I20260810 14:53:38.866537 6071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91ade0402f7848bd9f87aac726082454" candidate_uuid: "9562ca92f305486db348c6c91ed30016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd160ee324f54d4d9d942d1ed6215b31"
I20260810 14:53:38.866572 6216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91ade0402f7848bd9f87aac726082454" candidate_uuid: "9562ca92f305486db348c6c91ed30016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7835ad107dd430d838123f68e24a29b"
I20260810 14:53:38.866633 6071 raft_consensus.cc:3037] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:38.866671 6216 raft_consensus.cc:3037] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:38.867259 6071 raft_consensus.cc:2454] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9562ca92f305486db348c6c91ed30016 in term 1.
I20260810 14:53:38.867388 6216 raft_consensus.cc:2454] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9562ca92f305486db348c6c91ed30016 in term 1.
I20260810 14:53:38.867465 6292 leader_election.cc:302] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9562ca92f305486db348c6c91ed30016, dd160ee324f54d4d9d942d1ed6215b31; no voters:
I20260810 14:53:38.867571 6521 raft_consensus.cc:2781] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:38.867765 6521 raft_consensus.cc:686] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [term 1 LEADER]: Becoming Leader. State: Replica: 9562ca92f305486db348c6c91ed30016, State: Running, Role: LEADER
I20260810 14:53:38.867863 6521 consensus_queue.cc:237] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } } peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } }
I20260810 14:53:38.868129 6524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.868015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.67:36453)
0810 14:53:38.868065 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.868094 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 14:53:38.868649 5708 catalog_manager.cc:5495] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9562ca92f305486db348c6c91ed30016 (127.30.74.67). New cstate: current_term: 1 leader_uuid: "9562ca92f305486db348c6c91ed30016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9562ca92f305486db348c6c91ed30016" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 } health_report { overall_health: UNKNOWN } } }
I20260810 14:53:38.903030 6513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.900243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43368 (local address 127.30.74.67:36419)
0810 14:53:38.900335 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:38.900338 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:38.900348 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:38.900397 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:38.900398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:38.900417 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:38.900490 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:38.900626 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.901212 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.902174 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:38.902383 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:38.902467 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:38.902650 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:38.902760 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:38.902848 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:38.902921 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 14:53:38.902944 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:53:38.904307 6071 raft_consensus.cc:1261] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31 [term 1 FOLLOWER]: Refusing update from remote peer 9562ca92f305486db348c6c91ed30016: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:38.904423 6216 raft_consensus.cc:1261] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b [term 1 FOLLOWER]: Refusing update from remote peer 9562ca92f305486db348c6c91ed30016: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:38.904618 6521 consensus_queue.cc:1035] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd160ee324f54d4d9d942d1ed6215b31" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:38.904771 6525 consensus_queue.cc:1035] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7835ad107dd430d838123f68e24a29b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:38.905592 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.905474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.66:60093)
0810 14:53:38.905521 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.905555 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 14:53:38.906035 6495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.905873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.65:42587)
0810 14:53:38.905976 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.906003 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260810 14:53:38.907177 6524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.907036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.67:46367)
0810 14:53:38.907116 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.907147 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 14:53:38.907634 6524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.907484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.67:39391)
0810 14:53:38.907580 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.907606 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 14:53:38.907747 6496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.907306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.66:40161)
0810 14:53:38.907672 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.907712 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346}
I20260810 14:53:38.908413 6495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.908312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39103 (local address 127.30.74.65:53985)
0810 14:53:38.908349 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.908381 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260810 14:53:38.910012 5708 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:38.910117 5708 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:38.911403 5708 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fcc444f3a3e44c90b4aeb42775a3a4f3: Sending DeleteTablet for 3 replicas of tablet 91ade0402f7848bd9f87aac726082454
I20260810 14:53:38.911751 6340 tablet_service.cc:1514] Processing DeleteTablet for tablet 91ade0402f7848bd9f87aac726082454 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:38 UTC) from {username='oryx', principal='oryx/127.30.74.125@KRBTEST.COM'} at 127.0.0.1:43354
I20260810 14:53:38.911747 6050 tablet_service.cc:1514] Processing DeleteTablet for tablet 91ade0402f7848bd9f87aac726082454 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:38 UTC) from {username='oryx', principal='oryx/127.30.74.125@KRBTEST.COM'} at 127.0.0.1:59298
I20260810 14:53:38.911825 6195 tablet_service.cc:1514] Processing DeleteTablet for tablet 91ade0402f7848bd9f87aac726082454 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:38 UTC) from {username='oryx', principal='oryx/127.30.74.125@KRBTEST.COM'} at 127.0.0.1:58232
I20260810 14:53:38.911978 6540 tablet_replica.cc:330] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31: stopping tablet replica
I20260810 14:53:38.912025 6540 raft_consensus.cc:2229] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:38.912071 6540 raft_consensus.cc:2258] T 91ade0402f7848bd9f87aac726082454 P dd160ee324f54d4d9d942d1ed6215b31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:38.912169 6541 tablet_replica.cc:330] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b: stopping tablet replica
I20260810 14:53:38.912231 6541 raft_consensus.cc:2229] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:38.912315 6541 raft_consensus.cc:2258] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:38.912493 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 5871
I20260810 14:53:38.916474 6539 tablet_replica.cc:330] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016: stopping tablet replica
I20260810 14:53:38.916534 6541 ts_tablet_manager.cc:1905] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:38.916601 6539 raft_consensus.cc:2229] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:53:38.916743 6539 raft_consensus.cc:2258] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:38.917044 6539 ts_tablet_manager.cc:1905] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:38.918654 6539 ts_tablet_manager.cc:1918] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:38.918704 6541 ts_tablet_manager.cc:1918] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:38.918764 6539 log.cc:1196] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/91ade0402f7848bd9f87aac726082454
I20260810 14:53:38.918865 6541 log.cc:1196] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/91ade0402f7848bd9f87aac726082454
I20260810 14:53:38.919080 6539 ts_tablet_manager.cc:1939] T 91ade0402f7848bd9f87aac726082454 P 9562ca92f305486db348c6c91ed30016: Deleting consensus metadata
I20260810 14:53:38.919195 6541 ts_tablet_manager.cc:1939] T 91ade0402f7848bd9f87aac726082454 P a7835ad107dd430d838123f68e24a29b: Deleting consensus metadata
I20260810 14:53:38.919658 5692 catalog_manager.cc:4841] TS a7835ad107dd430d838123f68e24a29b (127.30.74.66:41633): tablet 91ade0402f7848bd9f87aac726082454 (table test-table [id=7952bda61ee24f8cbadc4098d67768cc]) successfully deleted
I20260810 14:53:38.919677 5693 catalog_manager.cc:4841] TS 9562ca92f305486db348c6c91ed30016 (127.30.74.67:36419): tablet 91ade0402f7848bd9f87aac726082454 (table test-table [id=7952bda61ee24f8cbadc4098d67768cc]) successfully deleted
I20260810 14:53:38.921190 6510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:38.921064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:35259 (local address 127.0.0.1:59304)
0810 14:53:38.921125 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:38.921158 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 14:53:38.921272 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 6128
W20260810 14:53:38.923238 5694 catalog_manager.cc:4584] TS dd160ee324f54d4d9d942d1ed6215b31 (127.30.74.65:35259): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 91ade0402f7848bd9f87aac726082454: Network error: Client connection negotiation failed: client connection to 127.30.74.65:35259: connect: Connection refused (error 111)
I20260810 14:53:38.936021 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 6272
I20260810 14:53:38.943187 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 5564
I20260810 14:53:38.951028 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 5768
2026-08-10T14:53:38Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6378 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:53:39 dist-test-slave-0b58 krb5kdc[6544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:53:39 dist-test-slave-0b58 krb5kdc[6544](info): set up 2 sockets
Aug 10 14:53:39 dist-test-slave-0b58 krb5kdc[6544](info): commencing operation
krb5kdc: starting...
W20260810 14:53:40.039772 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:53:40 dist-test-slave-0b58 krb5kdc[6544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:53:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:40Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.126.keytab.
Entry for principal oryx/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.126.keytab.
I20260810 14:53:40.167824 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:34909
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.126.keytab
--principal=oryx/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:34909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:40.254673 6557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:40.254916 6557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:40.254953 6557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:40.256109 6557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:40.256182 6557 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 14:53:40.256206 6557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:40.256239 6557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:40.256292 6557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:40.256317 6557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:40.257807 6557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:34909
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.126.keytab
--principal=oryx/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:34909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:40.257999 6557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:40.258178 6557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:40.262904 6564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:40.262948 6565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:40.263207 6557 server_base.cc:1034] running on GCE node
W20260810 14:53:40.262904 6567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:40.263600 6557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:40.263890 6557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:40.265081 6557 hybrid_clock.cc:648] HybridClock initialized: now 1786373620265067 us; error 31 us; skew 500 ppm
Aug 10 14:53:40 dist-test-slave-0b58 krb5kdc[6544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373620, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:40.267633 6557 init.cc:377] Logged in from keytab as oryx/127.30.74.126@KRBTEST.COM (short username oryx)
I20260810 14:53:40.268105 6557 webserver.cc:460] Webserver started at http://127.30.74.126:40329/ using document root <none> and password file <none>
I20260810 14:53:40.268378 6557 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:40.268427 6557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:40.268518 6557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:40.269547 6557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "b777745995174207ad875f7140c1a81e"
format_stamp: "Formatted at 2026-08-10 14:53:40 on dist-test-slave-0b58"
server_key: "29c1cf6cd040d96ac153879e6f7a1b0d"
server_key_iv: "7d828cc4b6aca14409d113c148f9f198"
server_key_version: "encryptionkey@0"
I20260810 14:53:40.269871 6557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "b777745995174207ad875f7140c1a81e"
format_stamp: "Formatted at 2026-08-10 14:53:40 on dist-test-slave-0b58"
server_key: "29c1cf6cd040d96ac153879e6f7a1b0d"
server_key_iv: "7d828cc4b6aca14409d113c148f9f198"
server_key_version: "encryptionkey@0"
I20260810 14:53:40.271155 6557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:53:40.271836 6574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:40.272017 6557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 14:53:40.272132 6557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "b777745995174207ad875f7140c1a81e"
format_stamp: "Formatted at 2026-08-10 14:53:40 on dist-test-slave-0b58"
server_key: "29c1cf6cd040d96ac153879e6f7a1b0d"
server_key_iv: "7d828cc4b6aca14409d113c148f9f198"
server_key_version: "encryptionkey@0"
I20260810 14:53:40.272214 6557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:40.280891 6557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:40.282724 6557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:40.282836 6557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:40.287578 6557 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:34909
I20260810 14:53:40.287623 6642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:34909 every 8 connection(s)
I20260810 14:53:40.287951 6557 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:40.288646 6643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:40.290935 6643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e: Bootstrap starting.
I20260810 14:53:40.291564 6643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:40.291868 6643 log.cc:824] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:40.292608 6643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e: No bootstrap required, opened a new log
I20260810 14:53:40.293251 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 6557
I20260810 14:53:40.293339 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:40.293560 31017 external_mini_cluster.cc:1428] Setting key 03ebe546fa6af340eb79adb445503127
I20260810 14:53:40.295338 6643 raft_consensus.cc:348] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b777745995174207ad875f7140c1a81e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34909 } }
I20260810 14:53:40.295517 6643 raft_consensus.cc:374] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:40.295557 6643 raft_consensus.cc:729] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b777745995174207ad875f7140c1a81e, State: Initialized, Role: FOLLOWER
I20260810 14:53:40.295719 6643 consensus_queue.cc:260] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b777745995174207ad875f7140c1a81e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34909 } }
I20260810 14:53:40.295789 6643 raft_consensus.cc:388] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:40.295810 6643 raft_consensus.cc:482] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:40.295881 6643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:40.296540 6643 raft_consensus.cc:504] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b777745995174207ad875f7140c1a81e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34909 } }
I20260810 14:53:40.296649 6643 leader_election.cc:302] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b777745995174207ad875f7140c1a81e; no voters:
I20260810 14:53:40.296813 6643 leader_election.cc:288] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:40.296880 6649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:40.297070 6643 sys_catalog.cc:564] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:40.297091 6649 raft_consensus.cc:686] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [term 1 LEADER]: Becoming Leader. State: Replica: b777745995174207ad875f7140c1a81e, State: Running, Role: LEADER
I20260810 14:53:40.297219 6649 consensus_queue.cc:237] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b777745995174207ad875f7140c1a81e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34909 } }
I20260810 14:53:40.297544 6650 sys_catalog.cc:455] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b777745995174207ad875f7140c1a81e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b777745995174207ad875f7140c1a81e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34909 } } }
I20260810 14:53:40.297534 6649 sys_catalog.cc:455] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b777745995174207ad875f7140c1a81e. Latest consensus state: current_term: 1 leader_uuid: "b777745995174207ad875f7140c1a81e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b777745995174207ad875f7140c1a81e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34909 } } }
I20260810 14:53:40.297646 6649 sys_catalog.cc:458] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:40.297638 6650 sys_catalog.cc:458] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:40.297884 6659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:40.298609 6659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:40.300340 6659 catalog_manager.cc:1347] Generated new cluster ID: e320b761cddf42f3a5ec094132592302
I20260810 14:53:40.300410 6659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 14:53:40 dist-test-slave-0b58 krb5kdc[6544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM
I20260810 14:53:40.307092 6659 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:40.307785 6659 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:40.308588 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.294136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52442 (local address 127.30.74.126:34909)
0810 14:53:40.294377 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:40.294387 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:40.294427 (+ 40us) server_negotiation.cc:407] Connection header received
0810 14:53:40.295294 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:40.295304 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:40.295370 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:40.295523 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:40.296319 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.297799 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.298928 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.299513 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.299632 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.302625 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:40.302631 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:40.302632 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:40.302652 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:40.306481 (+ 3829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:40.306968 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:40.306972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:40.306972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:40.307098 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:40.307428 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:40.307429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:40.307429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:40.307716 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:40.307922 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:40.308151 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 14:53:40.308351 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":97,"threads_started":1}
I20260810 14:53:40.313212 6659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e: Generated new TSK 0
I20260810 14:53:40.313432 6659 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.65.keytab.
Entry for principal oryx/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.65.keytab.
I20260810 14:53:40.339345 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:34909
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.65.keytab
--principal=oryx/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:40.424772 6671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:40.424968 6671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:40.424990 6671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:40.426113 6671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:40.426149 6671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:40.426175 6671 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:53:40.426193 6671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:53:40.427637 6671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.65.keytab
--principal=oryx/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.74.126:34909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:40.427826 6671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:40.428009 6671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:40.433096 6679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:40.433096 6676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:40.433099 6677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:40.433459 6671 server_base.cc:1034] running on GCE node
I20260810 14:53:40.433684 6671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:40.433894 6671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:40.435045 6671 hybrid_clock.cc:648] HybridClock initialized: now 1786373620435029 us; error 34 us; skew 500 ppm
Aug 10 14:53:40 dist-test-slave-0b58 krb5kdc[6544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373620, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:40.437794 6671 init.cc:377] Logged in from keytab as oryx/127.30.74.65@KRBTEST.COM (short username oryx)
I20260810 14:53:40.438299 6671 webserver.cc:460] Webserver started at http://127.30.74.65:33327/ using document root <none> and password file <none>
I20260810 14:53:40.438519 6671 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:40.438560 6671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:40.438651 6671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:40.439646 6671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e"
format_stamp: "Formatted at 2026-08-10 14:53:40 on dist-test-slave-0b58"
server_key: "ae86f09d149cca51b7a84c67c73528dd"
server_key_iv: "94f95249a5d5b4461ca9ea029d5444f7"
server_key_version: "encryptionkey@0"
I20260810 14:53:40.439949 6671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e"
format_stamp: "Formatted at 2026-08-10 14:53:40 on dist-test-slave-0b58"
server_key: "ae86f09d149cca51b7a84c67c73528dd"
server_key_iv: "94f95249a5d5b4461ca9ea029d5444f7"
server_key_version: "encryptionkey@0"
I20260810 14:53:40.441313 6671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:53:40.442023 6686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:40.442227 6671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:40.442341 6671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e"
format_stamp: "Formatted at 2026-08-10 14:53:40 on dist-test-slave-0b58"
server_key: "ae86f09d149cca51b7a84c67c73528dd"
server_key_iv: "94f95249a5d5b4461ca9ea029d5444f7"
server_key_version: "encryptionkey@0"
I20260810 14:53:40.442420 6671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:40.463218 6671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:40.465927 6671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:40.466122 6671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:40.466748 6671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:40.466809 6671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:40.466852 6671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:40.466873 6671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:53:40 dist-test-slave-0b58 krb5kdc[6544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373620, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM
I20260810 14:53:40.474761 6671 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:38925
I20260810 14:53:40.474860 6806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:38925 every 8 connection(s)
I20260810 14:53:40.475175 6671 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:53:40.478003 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.467179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:44919 (local address 127.30.74.126:34909)
0810 14:53:40.467267 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:40.467269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:40.467546 (+ 277us) server_negotiation.cc:407] Connection header received
0810 14:53:40.468338 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:40.468340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:40.468354 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:40.468424 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:40.469403 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.470015 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.471797 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.472095 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.472202 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.474857 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:40.474859 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:40.474859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:40.474868 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:40.476695 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:40.477091 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:40.477092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:40.477092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:40.477176 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:40.477405 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:40.477405 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:40.477405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:40.477513 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:40.477566 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:40.477867 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 14:53:40.477939 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 14:53:40.478199 6695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.467168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:34909 (local address 127.30.74.65:44919)
0810 14:53:40.467382 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:40.467442 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:40.468186 (+ 744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:40.468520 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:40.468525 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:40.468945 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:40.469265 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.469273 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.470125 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.470126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.471677 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.471682 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.472225 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.472226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.472381 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.472948 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:40.472956 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:40.474726 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:40.476772 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:40.476779 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:40.476780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:40.477018 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:40.477241 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:40.477241 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:40.477242 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:40.477346 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:40.477600 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:40.477606 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:40.477742 (+ 136us) client_negotiation.cc:769] Sending connection context
0810 14:53:40.477842 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 14:53:40.477977 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":112,"threads_started":1}
I20260810 14:53:40.478714 6807 heartbeater.cc:347] Connected to a master server at 127.30.74.126:34909
I20260810 14:53:40.478792 6807 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:40.478991 6807 heartbeater.cc:511] Master 127.30.74.126:34909 requested a full tablet report, sending...
I20260810 14:53:40.479626 6593 ts_manager.cc:194] Registered new tserver with Master: 579b52f91d6c42ebb7c7852a9ec1bd9e (127.30.74.65:38925)
I20260810 14:53:40.480389 6593 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:44919
I20260810 14:53:40.485316 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 6671
I20260810 14:53:40.485438 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:53:40.485538 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.479803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:35333 (local address 127.30.74.126:34909)
0810 14:53:40.479844 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:40.479846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:40.480000 (+ 154us) server_negotiation.cc:407] Connection header received
0810 14:53:40.480074 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:40.480076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:40.480086 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:40.480136 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:40.480568 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.481123 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.481844 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.482052 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.482138 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.483032 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:40.483033 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:40.483033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:40.483041 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:40.484374 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:40.484730 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:40.484731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:40.484731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:40.484793 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:40.485019 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:40.485019 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:40.485020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:40.485130 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:40.485202 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:40.485377 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 14:53:40.485448 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 14:53:40.485545 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.479807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:34909 (local address 127.30.74.65:35333)
0810 14:53:40.479930 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:40.479940 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:40.480016 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:40.480139 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:40.480140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:40.480390 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:40.480458 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.480460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.481226 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.481227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.481756 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.481758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.482142 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.482149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.482221 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.482640 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:40.482644 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:40.482918 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:40.484468 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:40.484470 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:40.484470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:40.484655 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:40.484871 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:40.484871 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:40.484872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:40.484942 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:40.485207 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:40.485208 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:40.485256 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 14:53:40.485337 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 14:53:40.485448 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
I20260810 14:53:40.485674 31017 external_mini_cluster.cc:1428] Setting key 84acdab73eb6e07b9d82664ded1f02f7
WARNING: no policy specified for oryx/127.30.74.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.66.keytab.
Entry for principal oryx/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.66.keytab.
I20260810 14:53:40.511919 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:34909
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.66.keytab
--principal=oryx/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:40.594856 6816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:40.595100 6816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:40.595144 6816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:40.596446 6816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:40.596509 6816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:40.596544 6816 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:53:40.596581 6816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:53:40.598316 6816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.66.keytab
--principal=oryx/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.74.126:34909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:40.598608 6816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:40.598851 6816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:40.603847 6821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:40.604025 6816 server_base.cc:1034] running on GCE node
W20260810 14:53:40.603858 6822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:40.603847 6824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:40.604417 6816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:40.604714 6816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:40.605880 6816 hybrid_clock.cc:648] HybridClock initialized: now 1786373620605857 us; error 36 us; skew 500 ppm
Aug 10 14:53:40 dist-test-slave-0b58 krb5kdc[6544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373620, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:40.608778 6816 init.cc:377] Logged in from keytab as oryx/127.30.74.66@KRBTEST.COM (short username oryx)
I20260810 14:53:40.609290 6816 webserver.cc:460] Webserver started at http://127.30.74.66:38459/ using document root <none> and password file <none>
I20260810 14:53:40.609534 6816 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:40.609604 6816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:40.609751 6816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:40.610863 6816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "efee7e9ba4fa4a1a9144f454d195c4ed"
format_stamp: "Formatted at 2026-08-10 14:53:40 on dist-test-slave-0b58"
server_key: "d87a85569698176cdcb991112a519ca7"
server_key_iv: "0c030d93dbf069c569898dbf1e142afa"
server_key_version: "encryptionkey@0"
I20260810 14:53:40.611266 6816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "efee7e9ba4fa4a1a9144f454d195c4ed"
format_stamp: "Formatted at 2026-08-10 14:53:40 on dist-test-slave-0b58"
server_key: "d87a85569698176cdcb991112a519ca7"
server_key_iv: "0c030d93dbf069c569898dbf1e142afa"
server_key_version: "encryptionkey@0"
I20260810 14:53:40.612761 6816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:53:40.613569 6831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:40.613744 6816 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 14:53:40.613863 6816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "efee7e9ba4fa4a1a9144f454d195c4ed"
format_stamp: "Formatted at 2026-08-10 14:53:40 on dist-test-slave-0b58"
server_key: "d87a85569698176cdcb991112a519ca7"
server_key_iv: "0c030d93dbf069c569898dbf1e142afa"
server_key_version: "encryptionkey@0"
I20260810 14:53:40.613947 6816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:40.633808 6816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:40.636415 6816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:40.636593 6816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:40.637216 6816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:40.637285 6816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:40.637334 6816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:40.637363 6816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:53:40 dist-test-slave-0b58 krb5kdc[6544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373620, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM
I20260810 14:53:40.644650 6816 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:44457
I20260810 14:53:40.644798 6952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:44457 every 8 connection(s)
I20260810 14:53:40.645090 6816 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:53:40.647253 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 6816
I20260810 14:53:40.647354 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:53:40.647606 31017 external_mini_cluster.cc:1428] Setting key f250af7cbcb23d46f693bb3b007bb68d
I20260810 14:53:40.648360 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.637821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:52849 (local address 127.30.74.126:34909)
0810 14:53:40.637888 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:40.637891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:40.638081 (+ 190us) server_negotiation.cc:407] Connection header received
0810 14:53:40.638664 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:40.638665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:40.638680 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:40.638739 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:40.639659 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.640533 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.642168 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.642499 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.642588 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.644926 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:40.644928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:40.644929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:40.644937 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:40.646672 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:40.647080 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:40.647081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:40.647082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:40.647152 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:40.647498 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:40.647499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:40.647500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:40.647657 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:40.647757 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:40.648142 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 14:53:40.648240 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 14:53:40.648545 6841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.637775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:34909 (local address 127.30.74.66:52849)
0810 14:53:40.637954 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:40.637991 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:40.638539 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:40.638821 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:40.638824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:40.639214 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:40.639534 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.639540 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.640644 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.640646 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.642067 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.642070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.642594 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.642595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.642705 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.643266 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:40.643276 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:40.644819 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:40.646753 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:40.646759 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:40.646760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:40.646986 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:40.647248 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:40.647249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:40.647249 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:40.647422 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:40.647792 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:40.647798 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:40.647992 (+ 194us) client_negotiation.cc:769] Sending connection context
0810 14:53:40.648117 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 14:53:40.648303 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260810 14:53:40.649070 6953 heartbeater.cc:347] Connected to a master server at 127.30.74.126:34909
I20260810 14:53:40.649169 6953 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:40.649374 6953 heartbeater.cc:511] Master 127.30.74.126:34909 requested a full tablet report, sending...
I20260810 14:53:40.649776 6593 ts_manager.cc:194] Registered new tserver with Master: efee7e9ba4fa4a1a9144f454d195c4ed (127.30.74.66:44457)
I20260810 14:53:40.650300 6593 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:52849
I20260810 14:53:40.656375 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.650131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:41635 (local address 127.30.74.126:34909)
0810 14:53:40.650165 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:40.650167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:40.650404 (+ 237us) server_negotiation.cc:407] Connection header received
0810 14:53:40.650656 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:40.650657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:40.650668 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:40.650723 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:40.651135 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.651625 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.652344 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.652664 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.652753 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.653603 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:40.653605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:40.653605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:40.653613 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:40.655186 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:40.655520 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:40.655521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:40.655522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:40.655588 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:40.655780 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:40.655781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:40.655781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:40.655979 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:40.656059 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:40.656167 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 14:53:40.656242 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 14:53:40.656387 6960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.650193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:34909 (local address 127.30.74.66:41635)
0810 14:53:40.650342 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:40.650353 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:40.650431 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:40.650767 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:40.650769 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:40.650973 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:40.651043 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.651045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.651720 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.651720 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.652214 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.652215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.652748 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.652749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.652797 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.653230 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:40.653234 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:40.653515 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:40.655278 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:40.655279 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:40.655279 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:40.655442 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:40.655658 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:40.655659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:40.655659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:40.655716 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:40.656052 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:40.656053 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:40.656088 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 14:53:40.656138 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 14:53:40.656240 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for oryx/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.67.keytab.
Entry for principal oryx/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.67.keytab.
I20260810 14:53:40.675216 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:34909
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.67.keytab
--principal=oryx/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:46591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:40.759780 6962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:40.759989 6962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:40.760012 6962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:40.761214 6962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:40.761263 6962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:40.761286 6962 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:53:40.761312 6962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:53:40.762836 6962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:46591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.67.keytab
--principal=oryx/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.74.126:34909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:40.763032 6962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:40.763211 6962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:40.768103 6970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:40.768139 6967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:40.768384 6962 server_base.cc:1034] running on GCE node
W20260810 14:53:40.768133 6968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:40.768689 6962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:40.768949 6962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:40.770093 6962 hybrid_clock.cc:648] HybridClock initialized: now 1786373620770082 us; error 33 us; skew 500 ppm
Aug 10 14:53:40 dist-test-slave-0b58 krb5kdc[6544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373620, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:40.772827 6962 init.cc:377] Logged in from keytab as oryx/127.30.74.67@KRBTEST.COM (short username oryx)
I20260810 14:53:40.773322 6962 webserver.cc:460] Webserver started at http://127.30.74.67:37085/ using document root <none> and password file <none>
I20260810 14:53:40.773561 6962 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:40.773627 6962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:40.773773 6962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:40.774881 6962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "f30d037ced4d47bfa34247b1980e9766"
format_stamp: "Formatted at 2026-08-10 14:53:40 on dist-test-slave-0b58"
server_key: "21ca049119cf0eb217a5f282b9475303"
server_key_iv: "618373f7183867262119fd2c8847368f"
server_key_version: "encryptionkey@0"
I20260810 14:53:40.775255 6962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "f30d037ced4d47bfa34247b1980e9766"
format_stamp: "Formatted at 2026-08-10 14:53:40 on dist-test-slave-0b58"
server_key: "21ca049119cf0eb217a5f282b9475303"
server_key_iv: "618373f7183867262119fd2c8847368f"
server_key_version: "encryptionkey@0"
I20260810 14:53:40.776813 6962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:53:40.777626 6980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:40.777858 6962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:40.777957 6962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "f30d037ced4d47bfa34247b1980e9766"
format_stamp: "Formatted at 2026-08-10 14:53:40 on dist-test-slave-0b58"
server_key: "21ca049119cf0eb217a5f282b9475303"
server_key_iv: "618373f7183867262119fd2c8847368f"
server_key_version: "encryptionkey@0"
I20260810 14:53:40.778034 6962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:40.795323 6962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:40.797976 6962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:40.798133 6962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:40.798781 6962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:40.798868 6962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:40.798910 6962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:40.798938 6962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:40.805224 6962 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:38573
I20260810 14:53:40.805720 7105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:38573 every 8 connection(s)
I20260810 14:53:40.806020 6962 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
Aug 10 14:53:40 dist-test-slave-0b58 krb5kdc[6544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373620, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM
I20260810 14:53:40.810561 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.799298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:60347 (local address 127.30.74.126:34909)
0810 14:53:40.799358 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:40.799360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:40.799591 (+ 231us) server_negotiation.cc:407] Connection header received
0810 14:53:40.800553 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:40.800555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:40.800571 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:40.800627 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:40.801657 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.802549 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.804364 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.804735 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.804814 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.807452 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:40.807453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:40.807454 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:40.807461 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:40.808852 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:40.809280 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:40.809281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:40.809281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:40.809344 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:40.809646 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:40.809648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:40.809648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:40.809816 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:40.809897 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:40.810312 (+ 415us) server_negotiation.cc:299] Negotiation successful
0810 14:53:40.810423 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 14:53:40.810729 6989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.799294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:34909 (local address 127.30.74.67:60347)
0810 14:53:40.799460 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:40.799506 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:40.800060 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:40.800713 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:40.800720 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:40.801156 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:40.801480 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.801487 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.802707 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.802708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.804201 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.804206 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.804826 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.804827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.805008 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.805693 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:40.805702 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:40.807341 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:40.808939 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:40.808946 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:40.808947 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:40.809171 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:40.809414 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:40.809415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:40.809415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:40.809548 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:40.809935 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:40.809943 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:40.810127 (+ 184us) client_negotiation.cc:769] Sending connection context
0810 14:53:40.810250 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 14:53:40.810484 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260810 14:53:40.811244 7106 heartbeater.cc:347] Connected to a master server at 127.30.74.126:34909
I20260810 14:53:40.811357 7106 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:40.811539 7106 heartbeater.cc:511] Master 127.30.74.126:34909 requested a full tablet report, sending...
I20260810 14:53:40.811810 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 6962
I20260810 14:53:40.811923 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:53:40.812135 31017 external_mini_cluster.cc:1428] Setting key 0be02ebb33e524983d8fd8a8936d7929
I20260810 14:53:40.812520 6593 ts_manager.cc:194] Registered new tserver with Master: f30d037ced4d47bfa34247b1980e9766 (127.30.74.67:38573)
I20260810 14:53:40.813063 6593 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:60347
I20260810 14:53:40.813076 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:53:40.819406 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.812554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:59387 (local address 127.30.74.126:34909)
0810 14:53:40.813192 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:40.813194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:40.813209 (+ 15us) server_negotiation.cc:407] Connection header received
0810 14:53:40.813227 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:40.813228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:40.813245 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:40.813300 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:40.813765 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.814267 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.815354 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.815669 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.815745 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.816823 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:40.816825 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:40.816825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:40.816832 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:40.818264 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:40.818619 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:40.818620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:40.818620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:40.818677 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:40.818886 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:40.818887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:40.818887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:40.818999 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:40.819076 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:40.819232 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 14:53:40.819311 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619}
I20260810 14:53:40.819420 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.812385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:34909 (local address 127.30.74.67:59387)
0810 14:53:40.812533 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:40.812544 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:40.812593 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:40.813353 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:40.813354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:40.813562 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:40.813655 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.813657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.814376 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.814377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.815255 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.815258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.815750 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.815751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.815807 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.816318 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:40.816323 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:40.816708 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:40.818363 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:40.818364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:40.818365 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:40.818544 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:40.818756 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:40.818757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:40.818757 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:40.818821 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:40.819069 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:40.819070 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:40.819117 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 14:53:40.819206 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 14:53:40.819313 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
Aug 10 14:53:40 dist-test-slave-0b58 krb5kdc[6544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373620, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 14:53:40 dist-test-slave-0b58 krb5kdc[6544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:40 dist-test-slave-0b58 krb5kdc[6544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 14:53:40.837725 6647 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:40.831845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48400 (local address 127.30.74.126:34909)
0810 14:53:40.831917 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:40.831919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:40.831930 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:53:40.831977 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:40.831978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:40.831988 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:40.832046 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:40.832728 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.833261 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.834010 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.834209 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.834268 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.837634 (+ 3366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48400: BlockingRecv error: recv got EOF from 127.0.0.1:48400 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 10 14:53:40 dist-test-slave-0b58 krb5kdc[6544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373620, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM
I20260810 14:53:40.846019 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.838717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48412 (local address 127.30.74.126:34909)
0810 14:53:40.838788 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:40.838791 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:40.838868 (+ 77us) server_negotiation.cc:407] Connection header received
0810 14:53:40.838952 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:40.838953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:40.838964 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:40.839014 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:40.839496 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.839992 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.840798 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.841008 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.841085 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.843379 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:40.843381 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:40.843381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:40.843389 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:40.844802 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:40.845184 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:40.845185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:40.845185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:40.845251 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:40.845455 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:40.845456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:40.845457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:40.845581 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:40.845693 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:40.845827 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 14:53:40.845913 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 14:53:40.847668 6593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:53:40.850142 6593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:40.857148 7131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.852769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:44457 (local address 127.0.0.1:47112)
0810 14:53:40.853165 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:40.853177 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:40.853243 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:40.853557 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:40.853561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:40.853570 (+ 9us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:40.853648 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.853649 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.854775 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.854777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.855921 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.855924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.856817 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.856818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.856913 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.856931 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 14:53:40.857009 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 14:53:40.857062 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":106,"threads_started":1}
I20260810 14:53:40.857187 7134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.853060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47112 (local address 127.30.74.66:44457)
0810 14:53:40.853352 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:40.853358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:40.853368 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:40.853420 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:40.853423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:40.853438 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:40.853525 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:40.853742 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.854663 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.856019 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.856723 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.856808 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.856902 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:40.857037 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 14:53:40.857078 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":91,"threads_started":1}
I20260810 14:53:40.857969 7129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.852687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:38925 (local address 127.0.0.1:55422)
0810 14:53:40.853085 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:40.853126 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:40.853208 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:40.853881 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:40.853882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:40.853884 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:40.853935 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.853937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.855323 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.855324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.856518 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.856521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.857687 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.857688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.857742 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.857745 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:53:40.857833 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 14:53:40.857881 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":33,"thread_start_us":47,"threads_started":1}
I20260810 14:53:40.858003 7132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.852855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55422 (local address 127.30.74.65:38925)
0810 14:53:40.853722 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:40.853726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:40.853738 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:53:40.853772 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:40.853774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:40.853786 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:40.853860 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:40.854051 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.855211 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.856615 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.857455 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.857556 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.857646 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:40.857855 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 14:53:40.857897 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":811,"thread_start_us":139,"threads_started":1}
I20260810 14:53:40.858083 6883 tablet_service.cc:1467] Processing CreateTablet for tablet f69c8c75082c46eea8c394616f0100be (DEFAULT_TABLE table=test-table [id=bb6b6e11e15d4a85a64f23939f776481]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:40.858523 6883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f69c8c75082c46eea8c394616f0100be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:40.861220 7130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.852687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:38573 (local address 127.0.0.1:48248)
0810 14:53:40.853072 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:40.853123 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:40.853209 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:40.854271 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:40.854273 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:40.854275 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:40.854320 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.854321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.855521 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.855522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.856623 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:40.856625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.860923 (+ 4298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.860924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:40.861014 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.861020 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:53:40.861094 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 14:53:40.861139 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":93,"threads_started":1}
I20260810 14:53:40.861050 6738 tablet_service.cc:1467] Processing CreateTablet for tablet f69c8c75082c46eea8c394616f0100be (DEFAULT_TABLE table=test-table [id=bb6b6e11e15d4a85a64f23939f776481]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:40.861492 6738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f69c8c75082c46eea8c394616f0100be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:40.861565 7133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:40.852935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48248 (local address 127.30.74.67:38573)
0810 14:53:40.853971 (+ 1036us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:40.853975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:40.853982 (+ 7us) server_negotiation.cc:407] Connection header received
0810 14:53:40.854055 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:40.854058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:40.854072 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:40.854192 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:40.854416 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.855440 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.856686 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:40.860823 (+ 4137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:40.860910 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:40.860998 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:40.861132 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 14:53:40.861186 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1004,"thread_start_us":124,"threads_started":1}
I20260810 14:53:40.862287 7135 tablet_bootstrap.cc:492] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed: Bootstrap starting.
I20260810 14:53:40.862241 7034 tablet_service.cc:1467] Processing CreateTablet for tablet f69c8c75082c46eea8c394616f0100be (DEFAULT_TABLE table=test-table [id=bb6b6e11e15d4a85a64f23939f776481]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:40.862598 7034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f69c8c75082c46eea8c394616f0100be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:40.863057 7135 tablet_bootstrap.cc:654] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:40.863421 7135 log.cc:824] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:40.865605 7137 tablet_bootstrap.cc:492] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766: Bootstrap starting.
I20260810 14:53:40.865890 7138 tablet_bootstrap.cc:492] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e: Bootstrap starting.
I20260810 14:53:40.866302 7137 tablet_bootstrap.cc:654] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:40.866622 7138 tablet_bootstrap.cc:654] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:40.866672 7135 tablet_bootstrap.cc:492] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed: No bootstrap required, opened a new log
I20260810 14:53:40.866643 7137 log.cc:824] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:40.866735 7135 ts_tablet_manager.cc:1397] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 14:53:40.867033 7138 log.cc:824] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:40.867511 7137 tablet_bootstrap.cc:492] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766: No bootstrap required, opened a new log
I20260810 14:53:40.867581 7137 ts_tablet_manager.cc:1397] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:53:40.867843 7138 tablet_bootstrap.cc:492] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e: No bootstrap required, opened a new log
I20260810 14:53:40.867935 7138 ts_tablet_manager.cc:1397] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:53:40.868887 7135 raft_consensus.cc:348] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:40.869072 7135 raft_consensus.cc:374] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:40.869112 7135 raft_consensus.cc:729] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efee7e9ba4fa4a1a9144f454d195c4ed, State: Initialized, Role: FOLLOWER
I20260810 14:53:40.869274 7135 consensus_queue.cc:260] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:40.869568 7135 ts_tablet_manager.cc:1428] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:53:40.869697 6953 heartbeater.cc:503] Master 127.30.74.126:34909 was elected leader, sending a full tablet report...
I20260810 14:53:40.869804 7137 raft_consensus.cc:348] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:40.869967 7137 raft_consensus.cc:374] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:40.870018 7137 raft_consensus.cc:729] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30d037ced4d47bfa34247b1980e9766, State: Initialized, Role: FOLLOWER
I20260810 14:53:40.870143 7137 consensus_queue.cc:260] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:40.870158 7138 raft_consensus.cc:348] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:40.870359 7138 raft_consensus.cc:374] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:40.870399 7137 ts_tablet_manager.cc:1428] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:53:40.870414 7138 raft_consensus.cc:729] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579b52f91d6c42ebb7c7852a9ec1bd9e, State: Initialized, Role: FOLLOWER
I20260810 14:53:40.870541 7138 consensus_queue.cc:260] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:40.870594 7106 heartbeater.cc:503] Master 127.30.74.126:34909 was elected leader, sending a full tablet report...
I20260810 14:53:40.870874 6807 heartbeater.cc:503] Master 127.30.74.126:34909 was elected leader, sending a full tablet report...
I20260810 14:53:40.870843 7138 ts_tablet_manager.cc:1428] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260810 14:53:40.895764 6954 tablet.cc:2369] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:53:40.976219 6808 tablet.cc:2369] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:53:41.015931 7142 raft_consensus.cc:482] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:41.016064 7142 raft_consensus.cc:504] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.016542 7142 leader_election.cc:288] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 579b52f91d6c42ebb7c7852a9ec1bd9e (127.30.74.65:38925), efee7e9ba4fa4a1a9144f454d195c4ed (127.30.74.66:44457)
I20260810 14:53:41.019448 7134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.016811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:42647 (local address 127.30.74.66:44457)
0810 14:53:41.016861 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.016863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.016875 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:53:41.016940 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.016941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.016950 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.017027 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:41.017182 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.017714 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.018720 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.019086 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.019173 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.019182 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:41.019340 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 14:53:41.019377 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 14:53:41.019469 6989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.016733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:44457 (local address 127.30.74.67:42647)
0810 14:53:41.016805 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:41.016816 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:41.016867 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:41.017032 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:41.017033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:41.017035 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:41.017107 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.017110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.017803 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.017804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.018623 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.018625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.019171 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.019172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.019263 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.019266 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:53:41.019334 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 14:53:41.019374 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 14:53:41.019564 7132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.016826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:48405 (local address 127.30.74.65:38925)
0810 14:53:41.016877 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.016879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.017004 (+ 125us) server_negotiation.cc:407] Connection header received
0810 14:53:41.017079 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.017080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.017090 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.017167 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:41.017386 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.017848 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.018792 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.019147 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.019235 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.019244 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:41.019444 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 14:53:41.019485 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 14:53:41.019790 7144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.016763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:38925 (local address 127.30.74.67:48405)
0810 14:53:41.016943 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:41.016953 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:41.017020 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:41.017171 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:41.017172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:41.017174 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:41.017259 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.017263 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.017936 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.017937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.018695 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.018697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.019236 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.019237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.019310 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.019314 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:41.019411 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 14:53:41.019456 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":131,"threads_started":1}
I20260810 14:53:41.019843 6906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f69c8c75082c46eea8c394616f0100be" candidate_uuid: "f30d037ced4d47bfa34247b1980e9766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" is_pre_election: true
I20260810 14:53:41.020074 6906 raft_consensus.cc:2454] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f30d037ced4d47bfa34247b1980e9766 in term 0.
I20260810 14:53:41.020161 6759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f69c8c75082c46eea8c394616f0100be" candidate_uuid: "f30d037ced4d47bfa34247b1980e9766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" is_pre_election: true
I20260810 14:53:41.020357 6759 raft_consensus.cc:2454] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f30d037ced4d47bfa34247b1980e9766 in term 0.
I20260810 14:53:41.020437 6981 leader_election.cc:302] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: efee7e9ba4fa4a1a9144f454d195c4ed, f30d037ced4d47bfa34247b1980e9766; no voters:
I20260810 14:53:41.020593 7142 raft_consensus.cc:2781] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:41.020661 7142 raft_consensus.cc:482] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:41.020697 7142 raft_consensus.cc:3037] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:41.021790 7142 raft_consensus.cc:504] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.021948 7142 leader_election.cc:288] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [CANDIDATE]: Term 1 election: Requested vote from peers 579b52f91d6c42ebb7c7852a9ec1bd9e (127.30.74.65:38925), efee7e9ba4fa4a1a9144f454d195c4ed (127.30.74.66:44457)
I20260810 14:53:41.022100 6759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f69c8c75082c46eea8c394616f0100be" candidate_uuid: "f30d037ced4d47bfa34247b1980e9766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e"
I20260810 14:53:41.022224 6906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f69c8c75082c46eea8c394616f0100be" candidate_uuid: "f30d037ced4d47bfa34247b1980e9766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed"
I20260810 14:53:41.022275 6759 raft_consensus.cc:3037] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:41.022307 6906 raft_consensus.cc:3037] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:41.023257 6759 raft_consensus.cc:2454] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f30d037ced4d47bfa34247b1980e9766 in term 1.
I20260810 14:53:41.023284 6906 raft_consensus.cc:2454] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f30d037ced4d47bfa34247b1980e9766 in term 1.
I20260810 14:53:41.023536 6982 leader_election.cc:302] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579b52f91d6c42ebb7c7852a9ec1bd9e, f30d037ced4d47bfa34247b1980e9766; no voters:
I20260810 14:53:41.023645 7142 raft_consensus.cc:2781] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:41.023792 7142 raft_consensus.cc:686] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [term 1 LEADER]: Becoming Leader. State: Replica: f30d037ced4d47bfa34247b1980e9766, State: Running, Role: LEADER
I20260810 14:53:41.023906 7142 consensus_queue.cc:237] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.024887 6593 catalog_manager.cc:5495] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 reported cstate change: term changed from 0 to 1, leader changed from <none> to f30d037ced4d47bfa34247b1980e9766 (127.30.74.67). New cstate: current_term: 1 leader_uuid: "f30d037ced4d47bfa34247b1980e9766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } health_report { overall_health: HEALTHY } } }
I20260810 14:53:41.038283 7133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.035859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48260 (local address 127.30.74.67:38573)
0810 14:53:41.035899 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.035901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.035910 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:41.035978 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.035979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.035995 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.036055 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:41.036186 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.036706 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.037552 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.037764 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.037822 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.038007 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:41.038102 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:41.038169 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:41.038211 (+ 42us) server_negotiation.cc:299] Negotiation successful
0810 14:53:41.038228 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 14:53:41.040171 6906 raft_consensus.cc:1261] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Refusing update from remote peer f30d037ced4d47bfa34247b1980e9766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:41.040177 6759 raft_consensus.cc:1261] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 FOLLOWER]: Refusing update from remote peer f30d037ced4d47bfa34247b1980e9766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:41.040573 7142 consensus_queue.cc:1035] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:41.040706 7145 consensus_queue.cc:1035] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:41.043403 7150 mvcc.cc:205] Tried to move back new op lower bound from 7316986351777894400 to 7316986351714717696. Current Snapshot: MvccSnapshot[applied={T|T < 7316986351777894400}]
I20260810 14:53:41.043905 7151 mvcc.cc:205] Tried to move back new op lower bound from 7316986351777894400 to 7316986351714717696. Current Snapshot: MvccSnapshot[applied={T|T < 7316986351777894400}]
I20260810 14:53:41.045591 6594 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:41.045684 6594 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:41.046516 6594 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e: Sending DeleteTablet for 3 replicas of tablet f69c8c75082c46eea8c394616f0100be
I20260810 14:53:41.046918 6738 tablet_service.cc:1514] Processing DeleteTablet for tablet f69c8c75082c46eea8c394616f0100be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:41 UTC) from {username='oryx', principal='oryx/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:55422
I20260810 14:53:41.046964 6883 tablet_service.cc:1514] Processing DeleteTablet for tablet f69c8c75082c46eea8c394616f0100be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:41 UTC) from {username='oryx', principal='oryx/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:47112
I20260810 14:53:41.046958 7034 tablet_service.cc:1514] Processing DeleteTablet for tablet f69c8c75082c46eea8c394616f0100be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:41 UTC) from {username='oryx', principal='oryx/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:48248
I20260810 14:53:41.047317 7159 tablet_replica.cc:330] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e: stopping tablet replica
I20260810 14:53:41.047433 7159 raft_consensus.cc:2229] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:41.047456 6594 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 14:53:41.047559 7159 raft_consensus.cc:2258] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 14:53:41.047607 6594 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:41.047986 7161 tablet_replica.cc:330] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed: stopping tablet replica
I20260810 14:53:41.048099 7161 raft_consensus.cc:2229] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:41.048235 7161 raft_consensus.cc:2258] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:41.048434 7159 ts_tablet_manager.cc:1905] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:41.048735 7161 ts_tablet_manager.cc:1905] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:41.049342 6738 tablet_service.cc:1467] Processing CreateTablet for tablet 223d7ed9f54e46eb8e3a8caf672ad233 (DEFAULT_TABLE table=test-table [id=b11b57a6ca894b368d52cc61980e0bd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:41.049471 6738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 223d7ed9f54e46eb8e3a8caf672ad233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:41.050506 7161 ts_tablet_manager.cc:1918] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:41.050601 7161 log.cc:1196] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/f69c8c75082c46eea8c394616f0100be
I20260810 14:53:41.050527 7159 ts_tablet_manager.cc:1918] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:41.050791 7159 log.cc:1196] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/wal/wals/f69c8c75082c46eea8c394616f0100be
I20260810 14:53:41.050843 7138 tablet_bootstrap.cc:492] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e: Bootstrap starting.
I20260810 14:53:41.050905 7161 ts_tablet_manager.cc:1939] T f69c8c75082c46eea8c394616f0100be P efee7e9ba4fa4a1a9144f454d195c4ed: Deleting consensus metadata
I20260810 14:53:41.051342 7159 ts_tablet_manager.cc:1939] T f69c8c75082c46eea8c394616f0100be P 579b52f91d6c42ebb7c7852a9ec1bd9e: Deleting consensus metadata
I20260810 14:53:41.051693 7034 tablet_service.cc:1467] Processing CreateTablet for tablet 223d7ed9f54e46eb8e3a8caf672ad233 (DEFAULT_TABLE table=test-table [id=b11b57a6ca894b368d52cc61980e0bd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:41.051746 6575 catalog_manager.cc:4841] TS efee7e9ba4fa4a1a9144f454d195c4ed (127.30.74.66:44457): tablet f69c8c75082c46eea8c394616f0100be (table test-table [id=bb6b6e11e15d4a85a64f23939f776481]) successfully deleted
I20260810 14:53:41.051826 7034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 223d7ed9f54e46eb8e3a8caf672ad233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:41.051925 6576 catalog_manager.cc:4841] TS 579b52f91d6c42ebb7c7852a9ec1bd9e (127.30.74.65:38925): tablet f69c8c75082c46eea8c394616f0100be (table test-table [id=bb6b6e11e15d4a85a64f23939f776481]) successfully deleted
I20260810 14:53:41.051402 7138 tablet_bootstrap.cc:654] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:41.052424 6883 tablet_service.cc:1467] Processing CreateTablet for tablet 223d7ed9f54e46eb8e3a8caf672ad233 (DEFAULT_TABLE table=test-table [id=b11b57a6ca894b368d52cc61980e0bd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:41.052541 6883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 223d7ed9f54e46eb8e3a8caf672ad233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:41.052636 7138 tablet_bootstrap.cc:492] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e: No bootstrap required, opened a new log
I20260810 14:53:41.052690 7138 ts_tablet_manager.cc:1397] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:53:41.052835 7138 raft_consensus.cc:348] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.052910 7138 raft_consensus.cc:374] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:41.052932 7138 raft_consensus.cc:729] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579b52f91d6c42ebb7c7852a9ec1bd9e, State: Initialized, Role: FOLLOWER
I20260810 14:53:41.052982 7138 consensus_queue.cc:260] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.053301 7138 ts_tablet_manager.cc:1428] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:41.053714 7135 tablet_bootstrap.cc:492] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed: Bootstrap starting.
I20260810 14:53:41.054142 7135 tablet_bootstrap.cc:654] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:41.054656 7137 tablet_bootstrap.cc:492] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766: Bootstrap starting.
I20260810 14:53:41.055035 7135 tablet_bootstrap.cc:492] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed: No bootstrap required, opened a new log
I20260810 14:53:41.055089 7135 ts_tablet_manager.cc:1397] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:41.055164 7137 tablet_bootstrap.cc:654] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:41.055250 7135 raft_consensus.cc:348] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.055325 7135 raft_consensus.cc:374] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:41.055348 7135 raft_consensus.cc:729] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efee7e9ba4fa4a1a9144f454d195c4ed, State: Initialized, Role: FOLLOWER
I20260810 14:53:41.055400 7135 consensus_queue.cc:260] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.055488 7135 ts_tablet_manager.cc:1428] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:41.055899 7137 tablet_bootstrap.cc:492] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766: No bootstrap required, opened a new log
I20260810 14:53:41.055953 7137 ts_tablet_manager.cc:1397] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:41.056090 7137 raft_consensus.cc:348] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.056164 7137 raft_consensus.cc:374] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:41.056185 7137 raft_consensus.cc:729] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30d037ced4d47bfa34247b1980e9766, State: Initialized, Role: FOLLOWER
I20260810 14:53:41.056226 7137 consensus_queue.cc:260] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.056342 7137 ts_tablet_manager.cc:1428] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:41.056358 7160 tablet_replica.cc:330] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766: stopping tablet replica
I20260810 14:53:41.056479 7160 raft_consensus.cc:2229] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [term 1 LEADER]: Raft consensus shutting down.
W20260810 14:53:41.056628 7107 tablet.cc:2369] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:53:41.056641 7160 raft_consensus.cc:2258] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:41.057104 7160 ts_tablet_manager.cc:1905] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:41.058605 7160 ts_tablet_manager.cc:1918] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:41.058692 7160 log.cc:1196] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/f69c8c75082c46eea8c394616f0100be
I20260810 14:53:41.058948 7160 ts_tablet_manager.cc:1939] T f69c8c75082c46eea8c394616f0100be P f30d037ced4d47bfa34247b1980e9766: Deleting consensus metadata
I20260810 14:53:41.059340 6578 catalog_manager.cc:4841] TS f30d037ced4d47bfa34247b1980e9766 (127.30.74.67:38573): tablet f69c8c75082c46eea8c394616f0100be (table test-table [id=bb6b6e11e15d4a85a64f23939f776481]) successfully deleted
I20260810 14:53:41.163915 7143 raft_consensus.cc:482] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:41.164045 7143 raft_consensus.cc:504] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.164455 7143 leader_election.cc:288] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efee7e9ba4fa4a1a9144f454d195c4ed (127.30.74.66:44457), f30d037ced4d47bfa34247b1980e9766 (127.30.74.67:38573)
I20260810 14:53:41.167526 7133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.164752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:46639 (local address 127.30.74.67:38573)
0810 14:53:41.164803 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.164805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.164943 (+ 138us) server_negotiation.cc:407] Connection header received
0810 14:53:41.165015 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.165016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.165028 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.165057 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:41.165269 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.165739 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.166798 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.167153 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.167247 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.167257 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:41.167416 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 14:53:41.167452 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 14:53:41.167527 7134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.164778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:45701 (local address 127.30.74.66:44457)
0810 14:53:41.164838 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.164841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.164986 (+ 145us) server_negotiation.cc:407] Connection header received
0810 14:53:41.165061 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.165063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.165072 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.165142 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:41.165308 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.165780 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.166794 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.167152 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.167243 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.167254 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:41.167420 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 14:53:41.167458 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 14:53:41.167558 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.164673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:38573 (local address 127.30.74.65:46639)
0810 14:53:41.164848 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:41.164860 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:41.164933 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:41.165124 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:41.165125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:41.165126 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:41.165189 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.165190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.165826 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.165827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.166700 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.166702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.167268 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.167269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.167341 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.167344 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:53:41.167412 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 14:53:41.167456 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":31,"thread_start_us":69,"threads_started":1}
I20260810 14:53:41.167562 7167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.164697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:44457 (local address 127.30.74.65:45701)
0810 14:53:41.164885 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:41.164895 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:41.164993 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:41.165146 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:41.165147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:41.165149 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:41.165224 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.165226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.165872 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.165873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.166697 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.166699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.167264 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.167265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.167344 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.167348 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:53:41.167414 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 14:53:41.167458 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260810 14:53:41.167824 6906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "223d7ed9f54e46eb8e3a8caf672ad233" candidate_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" is_pre_election: true
I20260810 14:53:41.167918 6906 raft_consensus.cc:2454] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579b52f91d6c42ebb7c7852a9ec1bd9e in term 0.
I20260810 14:53:41.167858 7054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "223d7ed9f54e46eb8e3a8caf672ad233" candidate_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30d037ced4d47bfa34247b1980e9766" is_pre_election: true
I20260810 14:53:41.168007 7054 raft_consensus.cc:2454] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579b52f91d6c42ebb7c7852a9ec1bd9e in term 0.
I20260810 14:53:41.168161 6687 leader_election.cc:302] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579b52f91d6c42ebb7c7852a9ec1bd9e, efee7e9ba4fa4a1a9144f454d195c4ed; no voters:
I20260810 14:53:41.168330 7143 raft_consensus.cc:2781] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:41.168394 7143 raft_consensus.cc:482] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:41.168427 7143 raft_consensus.cc:3037] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:41.169112 7143 raft_consensus.cc:504] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.169262 7143 leader_election.cc:288] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [CANDIDATE]: Term 1 election: Requested vote from peers efee7e9ba4fa4a1a9144f454d195c4ed (127.30.74.66:44457), f30d037ced4d47bfa34247b1980e9766 (127.30.74.67:38573)
I20260810 14:53:41.169436 7054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "223d7ed9f54e46eb8e3a8caf672ad233" candidate_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30d037ced4d47bfa34247b1980e9766"
I20260810 14:53:41.169509 7054 raft_consensus.cc:3037] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:41.169480 6906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "223d7ed9f54e46eb8e3a8caf672ad233" candidate_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed"
I20260810 14:53:41.169553 6906 raft_consensus.cc:3037] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:41.170310 7054 raft_consensus.cc:2454] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579b52f91d6c42ebb7c7852a9ec1bd9e in term 1.
I20260810 14:53:41.170342 6906 raft_consensus.cc:2454] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579b52f91d6c42ebb7c7852a9ec1bd9e in term 1.
I20260810 14:53:41.170495 6687 leader_election.cc:302] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579b52f91d6c42ebb7c7852a9ec1bd9e, efee7e9ba4fa4a1a9144f454d195c4ed; no voters:
I20260810 14:53:41.170591 7143 raft_consensus.cc:2781] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:41.170747 7143 raft_consensus.cc:686] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 LEADER]: Becoming Leader. State: Replica: 579b52f91d6c42ebb7c7852a9ec1bd9e, State: Running, Role: LEADER
I20260810 14:53:41.170851 7143 consensus_queue.cc:237] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.171566 6594 catalog_manager.cc:5495] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 579b52f91d6c42ebb7c7852a9ec1bd9e (127.30.74.65). New cstate: current_term: 1 leader_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } health_report { overall_health: UNKNOWN } } }
Aug 10 14:53:41 dist-test-slave-0b58 krb5kdc[6544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373620, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.126@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM
I20260810 14:53:41.227897 7176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.220985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:34909 (local address 127.0.0.1:48426)
0810 14:53:41.221163 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:41.221174 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:41.221244 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:41.221372 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:41.221373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:41.221705 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:41.221756 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.221759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.222528 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.222529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.223066 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.223067 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.223390 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.223391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.223435 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.224033 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:41.224038 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:41.225464 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:41.226892 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:41.226898 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:41.226901 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:41.227096 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:41.227281 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:41.227282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:41.227282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:41.227340 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:41.227583 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:41.227590 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:41.227639 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 14:53:41.227709 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 14:53:41.227820 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260810 14:53:41.227958 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.221063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48426 (local address 127.30.74.126:34909)
0810 14:53:41.221169 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.221171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.221243 (+ 72us) server_negotiation.cc:407] Connection header received
0810 14:53:41.221290 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.221291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.221300 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.221352 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:41.221824 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.222443 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.223142 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.223325 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.223374 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.225556 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:41.225557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:41.225557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:41.225563 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:41.226800 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:41.227174 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:41.227174 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:41.227175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:41.227222 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:41.227394 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:41.227394 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:41.227395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:41.227492 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:41.227543 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:41.227739 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 14:53:41.227873 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":71}
I20260810 14:53:41.230046 6594 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:48426:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 14:53:41.230253 6594 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:41.231818 6738 tablet_service.cc:1467] Processing CreateTablet for tablet b3b48e25a7e54ab2b6b91c1904ff355e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2cf82c4cbb2456ea71152dca6750f6f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:53:41.231814 6883 tablet_service.cc:1467] Processing CreateTablet for tablet b3b48e25a7e54ab2b6b91c1904ff355e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2cf82c4cbb2456ea71152dca6750f6f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:53:41.231921 7034 tablet_service.cc:1467] Processing CreateTablet for tablet b3b48e25a7e54ab2b6b91c1904ff355e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2cf82c4cbb2456ea71152dca6750f6f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:53:41.231997 6883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b48e25a7e54ab2b6b91c1904ff355e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:41.232007 6738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b48e25a7e54ab2b6b91c1904ff355e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:41.232062 7034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3b48e25a7e54ab2b6b91c1904ff355e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:41.233348 7138 tablet_bootstrap.cc:492] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e: Bootstrap starting.
I20260810 14:53:41.233589 7137 tablet_bootstrap.cc:492] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766: Bootstrap starting.
I20260810 14:53:41.233803 7138 tablet_bootstrap.cc:654] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:41.234022 7137 tablet_bootstrap.cc:654] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:41.234272 7135 tablet_bootstrap.cc:492] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed: Bootstrap starting.
I20260810 14:53:41.234668 7137 tablet_bootstrap.cc:492] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766: No bootstrap required, opened a new log
I20260810 14:53:41.234715 7135 tablet_bootstrap.cc:654] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:41.234715 7137 ts_tablet_manager.cc:1397] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:41.234840 7137 raft_consensus.cc:348] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.234951 7137 raft_consensus.cc:374] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:41.234974 7137 raft_consensus.cc:729] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30d037ced4d47bfa34247b1980e9766, State: Initialized, Role: FOLLOWER
I20260810 14:53:41.235028 7137 consensus_queue.cc:260] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.235134 7137 ts_tablet_manager.cc:1428] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:41.235234 7145 tablet_replica.cc:439] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } } }
I20260810 14:53:41.235324 7145 tablet_replica.cc:442] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:41.235673 7138 tablet_bootstrap.cc:492] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e: No bootstrap required, opened a new log
I20260810 14:53:41.235734 7138 ts_tablet_manager.cc:1397] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:53:41.235749 7135 tablet_bootstrap.cc:492] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed: No bootstrap required, opened a new log
I20260810 14:53:41.235805 7135 ts_tablet_manager.cc:1397] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:53:41.235884 7138 raft_consensus.cc:348] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.235968 7138 raft_consensus.cc:374] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:41.235996 7138 raft_consensus.cc:729] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579b52f91d6c42ebb7c7852a9ec1bd9e, State: Initialized, Role: FOLLOWER
I20260810 14:53:41.235970 7135 raft_consensus.cc:348] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.236061 7135 raft_consensus.cc:374] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:41.236053 7138 consensus_queue.cc:260] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.236093 7135 raft_consensus.cc:729] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efee7e9ba4fa4a1a9144f454d195c4ed, State: Initialized, Role: FOLLOWER
I20260810 14:53:41.236150 7138 ts_tablet_manager.cc:1428] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:41.236150 7135 consensus_queue.cc:260] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.236331 7135 ts_tablet_manager.cc:1428] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 14:53:41.236315 7143 tablet_replica.cc:439] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } } }
I20260810 14:53:41.236382 7143 tablet_replica.cc:442] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:41.236374 7141 tablet_replica.cc:439] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } } }
I20260810 14:53:41.236466 7141 tablet_replica.cc:442] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:41.336716 7141 raft_consensus.cc:482] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:41.336830 7141 raft_consensus.cc:504] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.337255 7141 leader_election.cc:288] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 579b52f91d6c42ebb7c7852a9ec1bd9e (127.30.74.65:38925), f30d037ced4d47bfa34247b1980e9766 (127.30.74.67:38573)
I20260810 14:53:41.340459 7132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.337568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:50139 (local address 127.30.74.65:38925)
0810 14:53:41.337630 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.337633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.337745 (+ 112us) server_negotiation.cc:407] Connection header received
0810 14:53:41.337867 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.337869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.337877 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.337948 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:41.338166 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.338680 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.339692 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.340039 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.340133 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.340144 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:41.340339 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 14:53:41.340378 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 14:53:41.340492 7133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.337596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:52899 (local address 127.30.74.67:38573)
0810 14:53:41.337645 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.337647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.337726 (+ 79us) server_negotiation.cc:407] Connection header received
0810 14:53:41.337792 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.337793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.337801 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.337886 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:53:41.338065 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.338651 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.339684 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.340070 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.340157 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.340166 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:41.340376 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 14:53:41.340421 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 14:53:41.340513 7181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.337486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:38573 (local address 127.30.74.66:52899)
0810 14:53:41.337626 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:41.337639 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:41.337715 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:41.337895 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:41.337897 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:41.337899 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:41.337980 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.337982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.338744 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.338745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.339582 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.339584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.340158 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.340159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.340300 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.340305 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:53:41.340356 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 14:53:41.340401 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
I20260810 14:53:41.340526 7182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.337486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:38925 (local address 127.30.74.66:50139)
0810 14:53:41.337661 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:41.337669 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:41.337775 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:41.337967 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:41.337968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:41.337971 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:53:41.338048 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.338049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.338766 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.338767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.339547 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.339549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.340144 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.340146 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.340220 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.340223 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:53:41.340330 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 14:53:41.340373 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":3,"thread_start_us":64,"threads_started":1}
I20260810 14:53:41.340742 7054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b48e25a7e54ab2b6b91c1904ff355e" candidate_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30d037ced4d47bfa34247b1980e9766" is_pre_election: true
I20260810 14:53:41.340843 7054 raft_consensus.cc:2454] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efee7e9ba4fa4a1a9144f454d195c4ed in term 0.
I20260810 14:53:41.340878 6759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b48e25a7e54ab2b6b91c1904ff355e" candidate_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" is_pre_election: true
I20260810 14:53:41.340999 6759 raft_consensus.cc:2454] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efee7e9ba4fa4a1a9144f454d195c4ed in term 0.
I20260810 14:53:41.341040 6835 leader_election.cc:302] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: efee7e9ba4fa4a1a9144f454d195c4ed, f30d037ced4d47bfa34247b1980e9766; no voters:
I20260810 14:53:41.341188 7141 raft_consensus.cc:2781] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:41.341239 7141 raft_consensus.cc:482] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:41.341254 7141 raft_consensus.cc:3037] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:41.341962 7141 raft_consensus.cc:504] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.342115 7141 leader_election.cc:288] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [CANDIDATE]: Term 1 election: Requested vote from peers 579b52f91d6c42ebb7c7852a9ec1bd9e (127.30.74.65:38925), f30d037ced4d47bfa34247b1980e9766 (127.30.74.67:38573)
I20260810 14:53:41.342296 7054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b48e25a7e54ab2b6b91c1904ff355e" candidate_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30d037ced4d47bfa34247b1980e9766"
I20260810 14:53:41.342338 6759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3b48e25a7e54ab2b6b91c1904ff355e" candidate_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e"
I20260810 14:53:41.342398 7054 raft_consensus.cc:3037] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:41.342417 6759 raft_consensus.cc:3037] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:41.343011 7054 raft_consensus.cc:2454] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efee7e9ba4fa4a1a9144f454d195c4ed in term 1.
I20260810 14:53:41.343153 6759 raft_consensus.cc:2454] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efee7e9ba4fa4a1a9144f454d195c4ed in term 1.
I20260810 14:53:41.343344 6833 leader_election.cc:302] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579b52f91d6c42ebb7c7852a9ec1bd9e, efee7e9ba4fa4a1a9144f454d195c4ed; no voters:
I20260810 14:53:41.343525 7141 raft_consensus.cc:2781] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:41.343684 7141 raft_consensus.cc:686] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 LEADER]: Becoming Leader. State: Replica: efee7e9ba4fa4a1a9144f454d195c4ed, State: Running, Role: LEADER
I20260810 14:53:41.343761 7141 consensus_queue.cc:237] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.344166 7183 tablet_replica.cc:439] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed: TxnStatusTablet state changed. Reason: New leader efee7e9ba4fa4a1a9144f454d195c4ed. Latest consensus state: current_term: 1 leader_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } } }
I20260810 14:53:41.344307 7183 tablet_replica.cc:442] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:53:41.344501 7185 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 14:53:41.344579 6593 catalog_manager.cc:5495] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed reported cstate change: term changed from 0 to 1, leader changed from <none> to efee7e9ba4fa4a1a9144f454d195c4ed (127.30.74.66). New cstate: current_term: 1 leader_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } health_report { overall_health: UNKNOWN } } }
I20260810 14:53:41.344669 7185 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 14:53:41.352437 7134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.349905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47128 (local address 127.30.74.66:44457)
0810 14:53:41.349989 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.349991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.350001 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:41.350115 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.350116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.350135 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.350200 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:41.350346 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.350828 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.351580 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.351793 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.351871 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.352064 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:41.352170 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:41.352252 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:41.352341 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 14:53:41.352363 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 14:53:41.352463 7176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.349842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:44457 (local address 127.0.0.1:47128)
0810 14:53:41.349948 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:41.349961 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:41.350013 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:41.350211 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:41.350212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:41.350213 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:53:41.350265 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.350267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.350916 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.350917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.351490 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.351491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.351870 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.351871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.351944 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.351967 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:53:41.352239 (+ 272us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:53:41.352249 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:53:41.352337 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 14:53:41.352391 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 14:53:41.353561 6759 raft_consensus.cc:1261] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 FOLLOWER]: Refusing update from remote peer efee7e9ba4fa4a1a9144f454d195c4ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:41.353667 7054 raft_consensus.cc:1261] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Refusing update from remote peer efee7e9ba4fa4a1a9144f454d195c4ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:41.353892 7183 consensus_queue.cc:1035] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:41.354040 7141 consensus_queue.cc:1035] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:41.354672 7145 tablet_replica.cc:439] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766: TxnStatusTablet state changed. Reason: New leader efee7e9ba4fa4a1a9144f454d195c4ed. Latest consensus state: current_term: 1 leader_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } } }
I20260810 14:53:41.354686 7143 tablet_replica.cc:439] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e: TxnStatusTablet state changed. Reason: New leader efee7e9ba4fa4a1a9144f454d195c4ed. Latest consensus state: current_term: 1 leader_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } } }
I20260810 14:53:41.354790 7143 tablet_replica.cc:442] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:41.354791 7145 tablet_replica.cc:442] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:41.355954 7183 tablet_replica.cc:439] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } } }
I20260810 14:53:41.356062 7183 tablet_replica.cc:442] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:53:41.356187 7149 mvcc.cc:205] Tried to move back new op lower bound from 7316986353062256640 to 7316986353024643072. Current Snapshot: MvccSnapshot[applied={T|T < 7316986353062256640}]
I20260810 14:53:41.356896 7145 tablet_replica.cc:439] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } } }
I20260810 14:53:41.356981 7145 tablet_replica.cc:442] T b3b48e25a7e54ab2b6b91c1904ff355e P f30d037ced4d47bfa34247b1980e9766: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:41.360427 7132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.357774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55424 (local address 127.30.74.65:38925)
0810 14:53:41.357826 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.357828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.357838 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:41.357899 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.357900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.357924 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.357998 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:41.358087 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.358833 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.359531 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.359863 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.359933 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.360061 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:41.360185 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:41.360280 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:41.360332 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 14:53:41.360354 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 14:53:41.364915 7183 tablet_replica.cc:439] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } } }
I20260810 14:53:41.365049 7183 tablet_replica.cc:442] T b3b48e25a7e54ab2b6b91c1904ff355e P efee7e9ba4fa4a1a9144f454d195c4ed: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:53:41.365407 7143 tablet_replica.cc:439] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } } }
I20260810 14:53:41.365511 7143 tablet_replica.cc:442] T b3b48e25a7e54ab2b6b91c1904ff355e P 579b52f91d6c42ebb7c7852a9ec1bd9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:53:41.366457 7134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.363048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:50417 (local address 127.30.74.66:44457)
0810 14:53:41.363122 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.363124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.363281 (+ 157us) server_negotiation.cc:407] Connection header received
0810 14:53:41.363338 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.363339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.363346 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.363391 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:41.363573 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.364075 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.365759 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.366039 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.366104 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.366285 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:41.366309 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:41.366342 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:41.366394 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 14:53:41.366409 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 14:53:41.366544 7193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.362965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:44457 (local address 127.30.74.65:50417)
0810 14:53:41.363205 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:41.363218 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:41.363281 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:41.363394 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:41.363394 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:41.363396 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:53:41.363460 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.363461 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.364158 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.364159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.365301 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.365304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.366114 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.366116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.366187 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.366204 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:53:41.366344 (+ 140us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:53:41.366345 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 14:53:41.366392 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 14:53:41.366435 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":99,"threads_started":1}
I20260810 14:53:41.368949 7054 raft_consensus.cc:1261] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Refusing update from remote peer 579b52f91d6c42ebb7c7852a9ec1bd9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:41.368949 6906 raft_consensus.cc:1261] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Refusing update from remote peer 579b52f91d6c42ebb7c7852a9ec1bd9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:41.369253 7168 consensus_queue.cc:1035] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:41.369387 7143 consensus_queue.cc:1035] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:41.379132 7132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.375712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:40595 (local address 127.30.74.65:38925)
0810 14:53:41.375766 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.375768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.375856 (+ 88us) server_negotiation.cc:407] Connection header received
0810 14:53:41.375946 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.375947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.375956 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.376055 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:41.376217 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.376989 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.378300 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.378552 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.378588 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.378836 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:41.378873 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:41.378957 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:41.379028 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 14:53:41.379050 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 14:53:41.379184 7199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.375629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:38925 (local address 127.30.74.66:40595)
0810 14:53:41.375792 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:41.375803 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:41.375883 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:41.376068 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:41.376069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:41.376071 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:53:41.376151 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.376153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.377088 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.377089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.378180 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.378184 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.378663 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.378664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.378718 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.378733 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:53:41.378959 (+ 226us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:53:41.378962 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:53:41.379022 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 14:53:41.379069 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260810 14:53:41.389326 6593 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:41.389446 6593 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:41.390100 6593 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e: Sending DeleteTablet for 3 replicas of tablet 223d7ed9f54e46eb8e3a8caf672ad233
I20260810 14:53:41.390357 6738 tablet_service.cc:1514] Processing DeleteTablet for tablet 223d7ed9f54e46eb8e3a8caf672ad233 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:41 UTC) from {username='oryx', principal='oryx/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:55422
I20260810 14:53:41.390407 7034 tablet_service.cc:1514] Processing DeleteTablet for tablet 223d7ed9f54e46eb8e3a8caf672ad233 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:41 UTC) from {username='oryx', principal='oryx/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:48248
I20260810 14:53:41.390463 7159 tablet_replica.cc:330] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e: stopping tablet replica
I20260810 14:53:41.390514 7159 raft_consensus.cc:2229] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:53:41.390599 7159 raft_consensus.cc:2258] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:41.390592 6883 tablet_service.cc:1514] Processing DeleteTablet for tablet 223d7ed9f54e46eb8e3a8caf672ad233 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:41 UTC) from {username='oryx', principal='oryx/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:47112
I20260810 14:53:41.390677 7160 tablet_replica.cc:330] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766: stopping tablet replica
I20260810 14:53:41.390704 7161 tablet_replica.cc:330] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed: stopping tablet replica
I20260810 14:53:41.390748 7160 raft_consensus.cc:2229] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:41.390751 7161 raft_consensus.cc:2229] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:41.390792 7161 raft_consensus.cc:2258] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:41.390795 7160 raft_consensus.cc:2258] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:41.391155 7161 ts_tablet_manager.cc:1905] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:41.391263 7160 ts_tablet_manager.cc:1905] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:41.392712 7160 ts_tablet_manager.cc:1918] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:53:41.392776 7160 log.cc:1196] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/223d7ed9f54e46eb8e3a8caf672ad233
I20260810 14:53:41.392823 7161 ts_tablet_manager.cc:1918] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:53:41.392876 7161 log.cc:1196] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/223d7ed9f54e46eb8e3a8caf672ad233
I20260810 14:53:41.393023 7160 ts_tablet_manager.cc:1939] T 223d7ed9f54e46eb8e3a8caf672ad233 P f30d037ced4d47bfa34247b1980e9766: Deleting consensus metadata
I20260810 14:53:41.393088 7161 ts_tablet_manager.cc:1939] T 223d7ed9f54e46eb8e3a8caf672ad233 P efee7e9ba4fa4a1a9144f454d195c4ed: Deleting consensus metadata
I20260810 14:53:41.393424 6578 catalog_manager.cc:4841] TS f30d037ced4d47bfa34247b1980e9766 (127.30.74.67:38573): tablet 223d7ed9f54e46eb8e3a8caf672ad233 (table test-table [id=b11b57a6ca894b368d52cc61980e0bd4]) successfully deleted
I20260810 14:53:41.393647 7159 ts_tablet_manager.cc:1905] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:41.395182 7159 ts_tablet_manager.cc:1918] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:53:41.395247 7159 log.cc:1196] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/wal/wals/223d7ed9f54e46eb8e3a8caf672ad233
I20260810 14:53:41.395453 7159 ts_tablet_manager.cc:1939] T 223d7ed9f54e46eb8e3a8caf672ad233 P 579b52f91d6c42ebb7c7852a9ec1bd9e: Deleting consensus metadata
I20260810 14:53:41.395794 6575 catalog_manager.cc:4841] TS efee7e9ba4fa4a1a9144f454d195c4ed (127.30.74.66:44457): tablet 223d7ed9f54e46eb8e3a8caf672ad233 (table test-table [id=b11b57a6ca894b368d52cc61980e0bd4]) successfully deleted
I20260810 14:53:41.395802 6576 catalog_manager.cc:4841] TS 579b52f91d6c42ebb7c7852a9ec1bd9e (127.30.74.65:38925): tablet 223d7ed9f54e46eb8e3a8caf672ad233 (table test-table [id=b11b57a6ca894b368d52cc61980e0bd4]) successfully deleted
Aug 10 14:53:41 dist-test-slave-0b58 krb5kdc[6544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 14:53:41 dist-test-slave-0b58 krb5kdc[6544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:41 dist-test-slave-0b58 krb5kdc[6544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 14:53:41.427639 6647 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:41.420474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48440 (local address 127.30.74.126:34909)
0810 14:53:41.420526 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.420529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.420718 (+ 189us) server_negotiation.cc:407] Connection header received
0810 14:53:41.420779 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.420780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.420793 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.420872 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:41.421636 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.422427 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.423611 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.423931 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.424005 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.427561 (+ 3556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48440: BlockingRecv error: recv got EOF from 127.0.0.1:48440 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 10 14:53:41 dist-test-slave-0b58 krb5kdc[6544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM
I20260810 14:53:41.440448 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.431343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48446 (local address 127.30.74.126:34909)
0810 14:53:41.431392 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.431394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.431488 (+ 94us) server_negotiation.cc:407] Connection header received
0810 14:53:41.431546 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.431547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.431558 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.431634 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:41.432285 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.433066 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.434114 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.434429 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.434504 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.437432 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:41.437434 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:41.437434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:41.437443 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:41.439125 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:41.439537 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:41.439538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:41.439539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:41.439607 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:41.439836 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:41.439837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:41.439837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:41.439983 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:41.440085 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:41.440236 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 14:53:41.440347 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:53:41.442551 6593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:53:41.442747 6593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:41.444242 7034 tablet_service.cc:1467] Processing CreateTablet for tablet 1b3b8388e3db4aad95aeb5d1d16328f8 (DEFAULT_TABLE table=test-table [id=3a38847c40f141e5b2dfdd7fc8aa39cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:41.444291 6883 tablet_service.cc:1467] Processing CreateTablet for tablet 1b3b8388e3db4aad95aeb5d1d16328f8 (DEFAULT_TABLE table=test-table [id=3a38847c40f141e5b2dfdd7fc8aa39cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:41.444419 7034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b3b8388e3db4aad95aeb5d1d16328f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:41.444432 6883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b3b8388e3db4aad95aeb5d1d16328f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:41.445977 7137 tablet_bootstrap.cc:492] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766: Bootstrap starting.
I20260810 14:53:41.446395 7137 tablet_bootstrap.cc:654] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:41.446727 7135 tablet_bootstrap.cc:492] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed: Bootstrap starting.
I20260810 14:53:41.447145 7135 tablet_bootstrap.cc:654] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:41.448000 6738 tablet_service.cc:1467] Processing CreateTablet for tablet 1b3b8388e3db4aad95aeb5d1d16328f8 (DEFAULT_TABLE table=test-table [id=3a38847c40f141e5b2dfdd7fc8aa39cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:41.448119 7137 tablet_bootstrap.cc:492] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766: No bootstrap required, opened a new log
I20260810 14:53:41.448133 6738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b3b8388e3db4aad95aeb5d1d16328f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:41.448166 7137 ts_tablet_manager.cc:1397] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:53:41.448385 7137 raft_consensus.cc:348] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.448465 7137 raft_consensus.cc:374] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:41.448487 7137 raft_consensus.cc:729] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30d037ced4d47bfa34247b1980e9766, State: Initialized, Role: FOLLOWER
I20260810 14:53:41.448532 7137 consensus_queue.cc:260] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.448622 7137 ts_tablet_manager.cc:1428] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:41.449136 7135 tablet_bootstrap.cc:492] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed: No bootstrap required, opened a new log
I20260810 14:53:41.449180 7135 ts_tablet_manager.cc:1397] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 14:53:41.449357 7135 raft_consensus.cc:348] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.449424 7135 raft_consensus.cc:374] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:41.449448 7135 raft_consensus.cc:729] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efee7e9ba4fa4a1a9144f454d195c4ed, State: Initialized, Role: FOLLOWER
I20260810 14:53:41.449491 7135 consensus_queue.cc:260] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.449592 7135 ts_tablet_manager.cc:1428] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:41.449736 7138 tablet_bootstrap.cc:492] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e: Bootstrap starting.
I20260810 14:53:41.450157 7138 tablet_bootstrap.cc:654] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:41.451320 7138 tablet_bootstrap.cc:492] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e: No bootstrap required, opened a new log
I20260810 14:53:41.451377 7138 ts_tablet_manager.cc:1397] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:53:41.451517 7138 raft_consensus.cc:348] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.451596 7138 raft_consensus.cc:374] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:41.451619 7138 raft_consensus.cc:729] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579b52f91d6c42ebb7c7852a9ec1bd9e, State: Initialized, Role: FOLLOWER
I20260810 14:53:41.451678 7138 consensus_queue.cc:260] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.451771 7138 ts_tablet_manager.cc:1428] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:41.682739 7143 raft_consensus.cc:482] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:41.682868 7143 raft_consensus.cc:504] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.683053 7143 leader_election.cc:288] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efee7e9ba4fa4a1a9144f454d195c4ed (127.30.74.66:44457), f30d037ced4d47bfa34247b1980e9766 (127.30.74.67:38573)
I20260810 14:53:41.683283 6906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b3b8388e3db4aad95aeb5d1d16328f8" candidate_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" is_pre_election: true
I20260810 14:53:41.683276 7054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b3b8388e3db4aad95aeb5d1d16328f8" candidate_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30d037ced4d47bfa34247b1980e9766" is_pre_election: true
I20260810 14:53:41.683386 6906 raft_consensus.cc:2454] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579b52f91d6c42ebb7c7852a9ec1bd9e in term 0.
I20260810 14:53:41.683398 7054 raft_consensus.cc:2454] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 579b52f91d6c42ebb7c7852a9ec1bd9e in term 0.
I20260810 14:53:41.683571 6687 leader_election.cc:302] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579b52f91d6c42ebb7c7852a9ec1bd9e, efee7e9ba4fa4a1a9144f454d195c4ed; no voters:
I20260810 14:53:41.683720 7143 raft_consensus.cc:2781] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:41.683753 7143 raft_consensus.cc:482] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:41.683771 7143 raft_consensus.cc:3037] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:41.684707 7143 raft_consensus.cc:504] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.684860 7143 leader_election.cc:288] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [CANDIDATE]: Term 1 election: Requested vote from peers efee7e9ba4fa4a1a9144f454d195c4ed (127.30.74.66:44457), f30d037ced4d47bfa34247b1980e9766 (127.30.74.67:38573)
I20260810 14:53:41.685020 6906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b3b8388e3db4aad95aeb5d1d16328f8" candidate_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed"
I20260810 14:53:41.685114 6906 raft_consensus.cc:3037] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:41.685149 7054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b3b8388e3db4aad95aeb5d1d16328f8" candidate_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30d037ced4d47bfa34247b1980e9766"
I20260810 14:53:41.685227 7054 raft_consensus.cc:3037] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:41.685986 7054 raft_consensus.cc:2454] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579b52f91d6c42ebb7c7852a9ec1bd9e in term 1.
I20260810 14:53:41.685986 6906 raft_consensus.cc:2454] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 579b52f91d6c42ebb7c7852a9ec1bd9e in term 1.
I20260810 14:53:41.686182 6690 leader_election.cc:302] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579b52f91d6c42ebb7c7852a9ec1bd9e, f30d037ced4d47bfa34247b1980e9766; no voters:
I20260810 14:53:41.686299 7143 raft_consensus.cc:2781] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:41.686370 7143 raft_consensus.cc:686] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 LEADER]: Becoming Leader. State: Replica: 579b52f91d6c42ebb7c7852a9ec1bd9e, State: Running, Role: LEADER
I20260810 14:53:41.686457 7143 consensus_queue.cc:237] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.687052 6592 catalog_manager.cc:5495] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 579b52f91d6c42ebb7c7852a9ec1bd9e (127.30.74.65). New cstate: current_term: 1 leader_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } health_report { overall_health: UNKNOWN } } }
I20260810 14:53:41.738284 7132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.735877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55434 (local address 127.30.74.65:38925)
0810 14:53:41.735952 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.735954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.735964 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:41.735993 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.735994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.736004 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.736055 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:41.736208 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.736742 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.737548 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.737763 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.737844 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.737992 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:41.738036 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:41.738092 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:41.738167 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 14:53:41.738191 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 14:53:41.739208 7054 raft_consensus.cc:1261] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Refusing update from remote peer 579b52f91d6c42ebb7c7852a9ec1bd9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:41.739223 6906 raft_consensus.cc:1261] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Refusing update from remote peer 579b52f91d6c42ebb7c7852a9ec1bd9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:41.739485 7143 consensus_queue.cc:1035] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:41.739596 7168 consensus_queue.cc:1035] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:41.742625 6593 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:41.742733 6593 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:41.743363 6593 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e: Sending DeleteTablet for 3 replicas of tablet 1b3b8388e3db4aad95aeb5d1d16328f8
I20260810 14:53:41.743633 6738 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b3b8388e3db4aad95aeb5d1d16328f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:41 UTC) from {username='oryx', principal='oryx/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:55422
I20260810 14:53:41.743674 6883 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b3b8388e3db4aad95aeb5d1d16328f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:41 UTC) from {username='oryx', principal='oryx/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:47112
I20260810 14:53:41.743672 7034 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b3b8388e3db4aad95aeb5d1d16328f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:41 UTC) from {username='oryx', principal='oryx/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:48248
I20260810 14:53:41.743767 7160 tablet_replica.cc:330] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766: stopping tablet replica
I20260810 14:53:41.743775 7159 tablet_replica.cc:330] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e: stopping tablet replica
I20260810 14:53:41.743824 7160 raft_consensus.cc:2229] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:41.743840 7159 raft_consensus.cc:2229] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:53:41.743870 7160 raft_consensus.cc:2258] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:41.743809 6593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 14:53:41.743901 7159 raft_consensus.cc:2258] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:41.743973 7161 tablet_replica.cc:330] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed: stopping tablet replica
W20260810 14:53:41.743971 6593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:53:41.744028 7161 raft_consensus.cc:2229] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:41.744077 7161 raft_consensus.cc:2258] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:41.744225 7160 ts_tablet_manager.cc:1905] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:41.744470 7161 ts_tablet_manager.cc:1905] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:41.745849 7160 ts_tablet_manager.cc:1918] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:41.745899 7161 ts_tablet_manager.cc:1918] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:41.745936 7160 log.cc:1196] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/1b3b8388e3db4aad95aeb5d1d16328f8
I20260810 14:53:41.745957 7161 log.cc:1196] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/1b3b8388e3db4aad95aeb5d1d16328f8
I20260810 14:53:41.746196 7160 ts_tablet_manager.cc:1939] T 1b3b8388e3db4aad95aeb5d1d16328f8 P f30d037ced4d47bfa34247b1980e9766: Deleting consensus metadata
I20260810 14:53:41.746223 7161 ts_tablet_manager.cc:1939] T 1b3b8388e3db4aad95aeb5d1d16328f8 P efee7e9ba4fa4a1a9144f454d195c4ed: Deleting consensus metadata
I20260810 14:53:41.746315 7159 ts_tablet_manager.cc:1905] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:41.746534 6578 catalog_manager.cc:4841] TS f30d037ced4d47bfa34247b1980e9766 (127.30.74.67:38573): tablet 1b3b8388e3db4aad95aeb5d1d16328f8 (table test-table [id=3a38847c40f141e5b2dfdd7fc8aa39cf]) successfully deleted
I20260810 14:53:41.746560 6575 catalog_manager.cc:4841] TS efee7e9ba4fa4a1a9144f454d195c4ed (127.30.74.66:44457): tablet 1b3b8388e3db4aad95aeb5d1d16328f8 (table test-table [id=3a38847c40f141e5b2dfdd7fc8aa39cf]) successfully deleted
I20260810 14:53:41.746973 6738 tablet_service.cc:1467] Processing CreateTablet for tablet fd0fd5918d034fd9871b45fe31e5425a (DEFAULT_TABLE table=test-table [id=6eeda74a06fc4aa5a46251b7e29d1fe1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:41.746973 6883 tablet_service.cc:1467] Processing CreateTablet for tablet fd0fd5918d034fd9871b45fe31e5425a (DEFAULT_TABLE table=test-table [id=6eeda74a06fc4aa5a46251b7e29d1fe1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:41.747084 6738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd0fd5918d034fd9871b45fe31e5425a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:41.747085 6883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd0fd5918d034fd9871b45fe31e5425a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:41.747649 7159 ts_tablet_manager.cc:1918] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:53:41.747690 7159 log.cc:1196] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-0/wal/wals/1b3b8388e3db4aad95aeb5d1d16328f8
I20260810 14:53:41.747850 7159 ts_tablet_manager.cc:1939] T 1b3b8388e3db4aad95aeb5d1d16328f8 P 579b52f91d6c42ebb7c7852a9ec1bd9e: Deleting consensus metadata
I20260810 14:53:41.748377 7034 tablet_service.cc:1467] Processing CreateTablet for tablet fd0fd5918d034fd9871b45fe31e5425a (DEFAULT_TABLE table=test-table [id=6eeda74a06fc4aa5a46251b7e29d1fe1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:53:41.748417 6576 catalog_manager.cc:4841] TS 579b52f91d6c42ebb7c7852a9ec1bd9e (127.30.74.65:38925): tablet 1b3b8388e3db4aad95aeb5d1d16328f8 (table test-table [id=3a38847c40f141e5b2dfdd7fc8aa39cf]) successfully deleted
I20260810 14:53:41.748442 7138 tablet_bootstrap.cc:492] T fd0fd5918d034fd9871b45fe31e5425a P 579b52f91d6c42ebb7c7852a9ec1bd9e: Bootstrap starting.
I20260810 14:53:41.748513 7034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd0fd5918d034fd9871b45fe31e5425a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:41.748767 7138 tablet_bootstrap.cc:654] T fd0fd5918d034fd9871b45fe31e5425a P 579b52f91d6c42ebb7c7852a9ec1bd9e: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:41.749471 7138 tablet_bootstrap.cc:492] T fd0fd5918d034fd9871b45fe31e5425a P 579b52f91d6c42ebb7c7852a9ec1bd9e: No bootstrap required, opened a new log
I20260810 14:53:41.749526 7138 ts_tablet_manager.cc:1397] T fd0fd5918d034fd9871b45fe31e5425a P 579b52f91d6c42ebb7c7852a9ec1bd9e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:41.749646 7138 raft_consensus.cc:348] T fd0fd5918d034fd9871b45fe31e5425a P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.749727 7138 raft_consensus.cc:374] T fd0fd5918d034fd9871b45fe31e5425a P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:41.749749 7138 raft_consensus.cc:729] T fd0fd5918d034fd9871b45fe31e5425a P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 579b52f91d6c42ebb7c7852a9ec1bd9e, State: Initialized, Role: FOLLOWER
I20260810 14:53:41.749800 7138 consensus_queue.cc:260] T fd0fd5918d034fd9871b45fe31e5425a P 579b52f91d6c42ebb7c7852a9ec1bd9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.749915 7138 ts_tablet_manager.cc:1428] T fd0fd5918d034fd9871b45fe31e5425a P 579b52f91d6c42ebb7c7852a9ec1bd9e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:41.750118 7135 tablet_bootstrap.cc:492] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed: Bootstrap starting.
I20260810 14:53:41.750159 7137 tablet_bootstrap.cc:492] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766: Bootstrap starting.
I20260810 14:53:41.750468 7135 tablet_bootstrap.cc:654] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:41.750466 7137 tablet_bootstrap.cc:654] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:41.751089 7135 tablet_bootstrap.cc:492] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed: No bootstrap required, opened a new log
I20260810 14:53:41.751151 7135 ts_tablet_manager.cc:1397] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:41.751401 7137 tablet_bootstrap.cc:492] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766: No bootstrap required, opened a new log
I20260810 14:53:41.751374 7135 raft_consensus.cc:348] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.751467 7137 ts_tablet_manager.cc:1397] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:41.751452 7135 raft_consensus.cc:374] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:41.751526 7135 raft_consensus.cc:729] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efee7e9ba4fa4a1a9144f454d195c4ed, State: Initialized, Role: FOLLOWER
I20260810 14:53:41.751585 7137 raft_consensus.cc:348] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.751588 7135 consensus_queue.cc:260] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.751679 7137 raft_consensus.cc:374] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:41.751713 7137 raft_consensus.cc:729] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30d037ced4d47bfa34247b1980e9766, State: Initialized, Role: FOLLOWER
I20260810 14:53:41.751735 7135 ts_tablet_manager.cc:1428] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:41.751762 7137 consensus_queue.cc:260] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.751864 7137 ts_tablet_manager.cc:1428] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:41.943629 7195 raft_consensus.cc:482] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:53:41.943744 7195 raft_consensus.cc:504] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.943984 7195 leader_election.cc:288] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 579b52f91d6c42ebb7c7852a9ec1bd9e (127.30.74.65:38925), f30d037ced4d47bfa34247b1980e9766 (127.30.74.67:38573)
I20260810 14:53:41.944195 6759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0fd5918d034fd9871b45fe31e5425a" candidate_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" is_pre_election: true
I20260810 14:53:41.944272 7054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0fd5918d034fd9871b45fe31e5425a" candidate_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30d037ced4d47bfa34247b1980e9766" is_pre_election: true
I20260810 14:53:41.944368 6759 raft_consensus.cc:2454] T fd0fd5918d034fd9871b45fe31e5425a P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efee7e9ba4fa4a1a9144f454d195c4ed in term 0.
I20260810 14:53:41.944406 7054 raft_consensus.cc:2454] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efee7e9ba4fa4a1a9144f454d195c4ed in term 0.
I20260810 14:53:41.944558 6833 leader_election.cc:302] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579b52f91d6c42ebb7c7852a9ec1bd9e, efee7e9ba4fa4a1a9144f454d195c4ed; no voters:
I20260810 14:53:41.944669 7195 raft_consensus.cc:2781] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:53:41.944721 7195 raft_consensus.cc:482] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:53:41.944736 7195 raft_consensus.cc:3037] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:41.945519 7195 raft_consensus.cc:504] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.945653 7195 leader_election.cc:288] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [CANDIDATE]: Term 1 election: Requested vote from peers 579b52f91d6c42ebb7c7852a9ec1bd9e (127.30.74.65:38925), f30d037ced4d47bfa34247b1980e9766 (127.30.74.67:38573)
I20260810 14:53:41.945825 6759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0fd5918d034fd9871b45fe31e5425a" candidate_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e"
I20260810 14:53:41.945899 6759 raft_consensus.cc:3037] T fd0fd5918d034fd9871b45fe31e5425a P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:41.945844 7054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0fd5918d034fd9871b45fe31e5425a" candidate_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f30d037ced4d47bfa34247b1980e9766"
I20260810 14:53:41.945946 7054 raft_consensus.cc:3037] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:41.946538 6759 raft_consensus.cc:2454] T fd0fd5918d034fd9871b45fe31e5425a P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efee7e9ba4fa4a1a9144f454d195c4ed in term 1.
I20260810 14:53:41.946707 6833 leader_election.cc:302] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 579b52f91d6c42ebb7c7852a9ec1bd9e, efee7e9ba4fa4a1a9144f454d195c4ed; no voters:
I20260810 14:53:41.946755 7054 raft_consensus.cc:2454] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efee7e9ba4fa4a1a9144f454d195c4ed in term 1.
I20260810 14:53:41.946810 7195 raft_consensus.cc:2781] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:41.946923 7195 raft_consensus.cc:686] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 LEADER]: Becoming Leader. State: Replica: efee7e9ba4fa4a1a9144f454d195c4ed, State: Running, Role: LEADER
I20260810 14:53:41.947005 7195 consensus_queue.cc:237] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } }
I20260810 14:53:41.947619 6592 catalog_manager.cc:5495] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed reported cstate change: term changed from 0 to 1, leader changed from <none> to efee7e9ba4fa4a1a9144f454d195c4ed (127.30.74.66). New cstate: current_term: 1 leader_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efee7e9ba4fa4a1a9144f454d195c4ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 44457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 } health_report { overall_health: UNKNOWN } } }
I20260810 14:53:41.979516 7229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.977026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47142 (local address 127.30.74.66:44457)
0810 14:53:41.977183 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.977185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.977193 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:53:41.977216 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.977216 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.977233 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.977319 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:41.977449 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.977951 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.978752 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.979010 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.979089 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.979237 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:41.979274 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:41.979346 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:41.979398 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 14:53:41.979421 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260810 14:53:41.984110 7229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.981925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:54657 (local address 127.30.74.66:44457)
0810 14:53:41.981977 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:41.981978 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:41.982104 (+ 126us) server_negotiation.cc:407] Connection header received
0810 14:53:41.982143 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:41.982143 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:41.982152 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:41.982226 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:53:41.982308 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.982744 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.983458 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.983632 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.983691 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.983800 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:53:41.983821 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:53:41.983854 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:41.984007 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 14:53:41.984030 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":31}
I20260810 14:53:41.984122 7232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:41.981873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:44457 (local address 127.30.74.66:54657)
0810 14:53:41.982035 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:41.982048 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:41.982103 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:41.982231 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:41.982231 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:41.982233 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:53:41.982272 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.982273 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.982812 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.982813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.983379 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:41.983380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:41.983706 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:41.983707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:41.983746 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:41.983750 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:53:41.983938 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:53:41.983941 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:53:41.983994 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 14:53:41.984042 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":2,"thread_start_us":111,"threads_started":1}
I20260810 14:53:41.986267 7054 raft_consensus.cc:1261] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Refusing update from remote peer efee7e9ba4fa4a1a9144f454d195c4ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:41.986474 7195 consensus_queue.cc:1035] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "f30d037ced4d47bfa34247b1980e9766" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:41.986601 6759 raft_consensus.cc:1261] T fd0fd5918d034fd9871b45fe31e5425a P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 FOLLOWER]: Refusing update from remote peer efee7e9ba4fa4a1a9144f454d195c4ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:53:41.987274 7228 consensus_queue.cc:1035] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "579b52f91d6c42ebb7c7852a9ec1bd9e" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:53:41.997913 6592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:41.998049 6592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:53:41.998729 6592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b777745995174207ad875f7140c1a81e: Sending DeleteTablet for 3 replicas of tablet fd0fd5918d034fd9871b45fe31e5425a
I20260810 14:53:41.999002 7034 tablet_service.cc:1514] Processing DeleteTablet for tablet fd0fd5918d034fd9871b45fe31e5425a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:41 UTC) from {username='oryx', principal='oryx/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:48248
I20260810 14:53:41.999128 7160 tablet_replica.cc:330] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766: stopping tablet replica
I20260810 14:53:41.999200 7160 raft_consensus.cc:2229] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:41.999244 7160 raft_consensus.cc:2258] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:41.999312 6738 tablet_service.cc:1514] Processing DeleteTablet for tablet fd0fd5918d034fd9871b45fe31e5425a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:41 UTC) from {username='oryx', principal='oryx/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:55422
I20260810 14:53:41.999414 7159 tablet_replica.cc:330] T fd0fd5918d034fd9871b45fe31e5425a P 579b52f91d6c42ebb7c7852a9ec1bd9e: stopping tablet replica
I20260810 14:53:41.999397 6882 tablet_service.cc:1514] Processing DeleteTablet for tablet fd0fd5918d034fd9871b45fe31e5425a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:53:41 UTC) from {username='oryx', principal='oryx/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:47112
I20260810 14:53:41.999475 7159 raft_consensus.cc:2229] T fd0fd5918d034fd9871b45fe31e5425a P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:53:41.999497 7161 tablet_replica.cc:330] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed: stopping tablet replica
I20260810 14:53:41.999516 7159 raft_consensus.cc:2258] T fd0fd5918d034fd9871b45fe31e5425a P 579b52f91d6c42ebb7c7852a9ec1bd9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:41.999552 7161 raft_consensus.cc:2229] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:53:41.999635 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 6671
I20260810 14:53:41.999660 7161 raft_consensus.cc:2258] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:53:41.999733 7160 ts_tablet_manager.cc:1905] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:42.001425 7160 ts_tablet_manager.cc:1918] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:53:42.001523 7160 log.cc:1196] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/fd0fd5918d034fd9871b45fe31e5425a
I20260810 14:53:42.001756 7160 ts_tablet_manager.cc:1939] T fd0fd5918d034fd9871b45fe31e5425a P f30d037ced4d47bfa34247b1980e9766: Deleting consensus metadata
I20260810 14:53:42.002103 6578 catalog_manager.cc:4841] TS f30d037ced4d47bfa34247b1980e9766 (127.30.74.67:38573): tablet fd0fd5918d034fd9871b45fe31e5425a (table test-table [id=6eeda74a06fc4aa5a46251b7e29d1fe1]) successfully deleted
I20260810 14:53:42.005488 7161 ts_tablet_manager.cc:1905] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:53:42.007447 7161 ts_tablet_manager.cc:1918] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:53:42.007520 7161 log.cc:1196] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/fd0fd5918d034fd9871b45fe31e5425a
I20260810 14:53:42.007738 7161 ts_tablet_manager.cc:1939] T fd0fd5918d034fd9871b45fe31e5425a P efee7e9ba4fa4a1a9144f454d195c4ed: Deleting consensus metadata
I20260810 14:53:42.008231 6575 catalog_manager.cc:4841] TS efee7e9ba4fa4a1a9144f454d195c4ed (127.30.74.66:44457): tablet fd0fd5918d034fd9871b45fe31e5425a (table test-table [id=6eeda74a06fc4aa5a46251b7e29d1fe1]) successfully deleted
W20260810 14:53:42.011917 6576 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:38925 (error 108)
I20260810 14:53:42.012243 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 6816
I20260810 14:53:42.020551 7238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:42.012359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:38925 (local address 127.0.0.1:55444)
0810 14:53:42.020424 (+ 8065us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:42.020501 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:38925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8040,"thread_start_us":48,"threads_started":1}
W20260810 14:53:42.020766 6576 catalog_manager.cc:4584] TS 579b52f91d6c42ebb7c7852a9ec1bd9e (127.30.74.65:38925): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fd0fd5918d034fd9871b45fe31e5425a: Network error: Client connection negotiation failed: client connection to 127.30.74.65:38925: connect: Connection refused (error 111)
I20260810 14:53:42.022183 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 6962
I20260810 14:53:42.031236 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 6557
2026-08-10T14:53:42Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3080 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:53:42 dist-test-slave-0b58 krb5kdc[7241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:53:42 dist-test-slave-0b58 krb5kdc[7241](info): set up 2 sockets
Aug 10 14:53:42 dist-test-slave-0b58 krb5kdc[7241](info): commencing operation
krb5kdc: starting...
W20260810 14:53:43.120406 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:53:43 dist-test-slave-0b58 krb5kdc[7241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:53:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:53:43.248771 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:33477
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:44591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:33477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:43.332527 7255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:43.332742 7255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:43.332768 7255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:43.334028 7255 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:43.334072 7255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:43.334086 7255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:43.334100 7255 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:43.334111 7255 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:43.335632 7255 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:44591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:33477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:33477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:43.335894 7255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:43.336167 7255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:43.341004 7261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:43.341274 7260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:43.341367 7255 server_base.cc:1034] running on GCE node
W20260810 14:53:43.341504 7263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:43.341758 7255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:43.342062 7255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:43.343237 7255 hybrid_clock.cc:648] HybridClock initialized: now 1786373623343230 us; error 37 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T14:53:43Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1312 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:53:43 dist-test-slave-0b58 krb5kdc[7269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:53:43 dist-test-slave-0b58 krb5kdc[7269](info): set up 2 sockets
Aug 10 14:53:43 dist-test-slave-0b58 krb5kdc[7269](info): commencing operation
krb5kdc: starting...
W20260810 14:53:44.429091 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:53:44 dist-test-slave-0b58 krb5kdc[7269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:53:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:53:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:53:44.557173 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:42265
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:39597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:42265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:44.640065 7283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:44.640307 7283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:44.640337 7283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:44.641559 7283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:44.641616 7283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:44.641631 7283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:44.641644 7283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:44.641656 7283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:44.643324 7283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:39597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:42265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:42265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:44.643603 7283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:44.643855 7283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:44.648872 7289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:44.648907 7288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:44.649096 7283 server_base.cc:1034] running on GCE node
W20260810 14:53:44.648907 7291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:44.649453 7283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:44.649705 7283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:44.650897 7283 hybrid_clock.cc:648] HybridClock initialized: now 1786373624650884 us; error 29 us; skew 500 ppm
Aug 10 14:53:44 dist-test-slave-0b58 krb5kdc[7269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373624, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:44.661216 7283 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:53:44.661819 7283 webserver.cc:460] Webserver started at http://127.30.74.126:44269/ using document root <none> and password file <none>
I20260810 14:53:44.662103 7283 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:44.662189 7283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:44.662344 7283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:44.663488 7283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "db2bccbc02bf41358d4d6da9df4dba62"
format_stamp: "Formatted at 2026-08-10 14:53:44 on dist-test-slave-0b58"
server_key: "57776d0dd624c16ce2aba54b1b41c9e3"
server_key_iv: "c0cf7ba0382e102fec9033170864ba37"
server_key_version: "encryptionkey@0"
I20260810 14:53:44.663873 7283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "db2bccbc02bf41358d4d6da9df4dba62"
format_stamp: "Formatted at 2026-08-10 14:53:44 on dist-test-slave-0b58"
server_key: "57776d0dd624c16ce2aba54b1b41c9e3"
server_key_iv: "c0cf7ba0382e102fec9033170864ba37"
server_key_version: "encryptionkey@0"
I20260810 14:53:44.665380 7283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:53:44.666275 7298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:44.666505 7283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:44.666602 7283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "db2bccbc02bf41358d4d6da9df4dba62"
format_stamp: "Formatted at 2026-08-10 14:53:44 on dist-test-slave-0b58"
server_key: "57776d0dd624c16ce2aba54b1b41c9e3"
server_key_iv: "c0cf7ba0382e102fec9033170864ba37"
server_key_version: "encryptionkey@0"
I20260810 14:53:44.666690 7283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:44.678650 7283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:44.681082 7283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:44.681272 7283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:44.685622 7283 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:42265
I20260810 14:53:44.685678 7354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:42265 every 8 connection(s)
I20260810 14:53:44.686059 7283 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:44.686965 7355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:53:44.689363 7355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62: Bootstrap starting.
I20260810 14:53:44.689985 7355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62: Neither blocks nor log segments found. Creating new log.
I20260810 14:53:44.690306 7355 log.cc:824] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:44.691033 7355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62: No bootstrap required, opened a new log
I20260810 14:53:44.692703 7355 raft_consensus.cc:348] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2bccbc02bf41358d4d6da9df4dba62" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42265 } }
I20260810 14:53:44.692870 7355 raft_consensus.cc:374] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:53:44.692971 7355 raft_consensus.cc:729] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2bccbc02bf41358d4d6da9df4dba62, State: Initialized, Role: FOLLOWER
I20260810 14:53:44.693145 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 7283
I20260810 14:53:44.693182 7355 consensus_queue.cc:260] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2bccbc02bf41358d4d6da9df4dba62" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42265 } }
I20260810 14:53:44.693276 7355 raft_consensus.cc:388] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:44.693295 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:53:44.693352 7355 raft_consensus.cc:482] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:44.693425 7355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:53:44.693490 31017 external_mini_cluster.cc:1428] Setting key 7d5d4727fc0eeb46c8818f61316be3c9
I20260810 14:53:44.694247 7355 raft_consensus.cc:504] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2bccbc02bf41358d4d6da9df4dba62" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42265 } }
I20260810 14:53:44.694401 7355 leader_election.cc:302] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db2bccbc02bf41358d4d6da9df4dba62; no voters:
I20260810 14:53:44.694617 7355 leader_election.cc:288] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:53:44.694811 7362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:53:44.694902 7355 sys_catalog.cc:564] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:44.695007 7362 raft_consensus.cc:686] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 1 LEADER]: Becoming Leader. State: Replica: db2bccbc02bf41358d4d6da9df4dba62, State: Running, Role: LEADER
I20260810 14:53:44.695173 7362 consensus_queue.cc:237] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2bccbc02bf41358d4d6da9df4dba62" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42265 } }
I20260810 14:53:44.695521 7362 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db2bccbc02bf41358d4d6da9df4dba62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2bccbc02bf41358d4d6da9df4dba62" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42265 } } }
I20260810 14:53:44.695675 7362 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:44.695899 7362 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db2bccbc02bf41358d4d6da9df4dba62. Latest consensus state: current_term: 1 leader_uuid: "db2bccbc02bf41358d4d6da9df4dba62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2bccbc02bf41358d4d6da9df4dba62" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42265 } } }
I20260810 14:53:44.695976 7370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:44.696031 7362 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:44.696592 7370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 14:53:44 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:44.707690 7370 catalog_manager.cc:1347] Generated new cluster ID: 095798fd05594c3c83719050aff84d7b
I20260810 14:53:44.707763 7370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:53:44.710403 7359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:44.694412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47774 (local address 127.30.74.126:42265)
0810 14:53:44.694637 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:44.694649 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:44.694688 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:53:44.694850 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:44.694857 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:44.694914 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:44.695061 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:44.697212 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:44.698647 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:44.699780 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:44.700438 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:44.700548 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:44.706233 (+ 5685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:44.706239 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:44.706239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:44.706253 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:44.708717 (+ 2464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:44.709141 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:44.709144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:44.709144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:44.709225 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:44.709460 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:44.709460 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:44.709461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:44.709717 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:44.709884 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:44.710008 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 14:53:44.710153 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":126,"threads_started":1}
I20260810 14:53:44.726588 7370 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:53:44.727144 7370 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:44.731097 7370 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62: Generated new TSK 0
I20260810 14:53:44.731307 7370 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:53:44.763286 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42265
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:39597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:44.846763 7382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:44.847009 7382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:44.847050 7382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:44.848308 7382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:44.848377 7382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:44.848424 7382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:53:44.849905 7382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:39597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:44.850193 7382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:44.850437 7382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:44.855500 7390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:44.855506 7389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:44.855480 7392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:44.855711 7382 server_base.cc:1034] running on GCE node
I20260810 14:53:44.855993 7382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:44.856318 7382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:44.857481 7382 hybrid_clock.cc:648] HybridClock initialized: now 1786373624857467 us; error 46 us; skew 500 ppm
Aug 10 14:53:44 dist-test-slave-0b58 krb5kdc[7269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373624, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:44.860033 7382 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:53:44.860558 7382 webserver.cc:460] Webserver started at http://127.30.74.65:40211/ using document root <none> and password file <none>
I20260810 14:53:44.860805 7382 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:44.860875 7382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:44.861023 7382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:44.862176 7382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "2d1f95c9eb00412aac7a87daf5d4de34"
format_stamp: "Formatted at 2026-08-10 14:53:44 on dist-test-slave-0b58"
server_key: "8f4cbb539b8348af26d58c8ed0ee7533"
server_key_iv: "b692a0463b7b5408de9bfadadfb8c824"
server_key_version: "encryptionkey@0"
I20260810 14:53:44.862547 7382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "2d1f95c9eb00412aac7a87daf5d4de34"
format_stamp: "Formatted at 2026-08-10 14:53:44 on dist-test-slave-0b58"
server_key: "8f4cbb539b8348af26d58c8ed0ee7533"
server_key_iv: "b692a0463b7b5408de9bfadadfb8c824"
server_key_version: "encryptionkey@0"
I20260810 14:53:44.863910 7382 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 14:53:44.864745 7400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:44.864921 7382 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:44.865032 7382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "2d1f95c9eb00412aac7a87daf5d4de34"
format_stamp: "Formatted at 2026-08-10 14:53:44 on dist-test-slave-0b58"
server_key: "8f4cbb539b8348af26d58c8ed0ee7533"
server_key_iv: "b692a0463b7b5408de9bfadadfb8c824"
server_key_version: "encryptionkey@0"
I20260810 14:53:44.865110 7382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:44.891742 7382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:44.893791 7382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:44.893963 7382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:44.894201 7382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:44.894536 7382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:44.894593 7382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:44.894644 7382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:44.894691 7382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:44.901021 7382 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:39143
I20260810 14:53:44.901095 7517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:39143 every 8 connection(s)
I20260810 14:53:44.901475 7382 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
Aug 10 14:53:44 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373624, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:44.909603 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 7382
I20260810 14:53:44.909729 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:53:44.909953 31017 external_mini_cluster.cc:1428] Setting key a5669179b1a962850cffa6a4fac45f19
I20260810 14:53:44.911722 7359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:44.902562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:35689 (local address 127.30.74.126:42265)
0810 14:53:44.902623 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:44.902625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:44.902845 (+ 220us) server_negotiation.cc:407] Connection header received
0810 14:53:44.903333 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:44.903334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:44.903346 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:44.903402 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:44.904358 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:44.904867 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:44.905859 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:44.906050 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:44.906117 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:44.908377 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:44.908379 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:44.908379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:44.908387 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:44.910141 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:44.910587 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:44.910588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:44.910589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:44.910676 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:44.910957 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:44.910958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:44.910958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:44.911109 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:44.911183 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:44.911484 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 14:53:44.911590 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 14:53:44.911738 7520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:44.902512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:35689)
0810 14:53:44.902714 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:44.902758 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:44.903252 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:44.903460 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:44.903463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:44.904027 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:44.904229 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:44.904234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:44.904947 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:44.904948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:44.905782 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:44.905784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:44.906160 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:44.906161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:44.906314 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:44.906800 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:44.906808 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:44.908221 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:44.910252 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:44.910266 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:44.910268 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:44.910503 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:44.910756 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:44.910757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:44.910757 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:44.910881 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:44.911208 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:44.911213 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:44.911341 (+ 128us) client_negotiation.cc:769] Sending connection context
0810 14:53:44.911438 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 14:53:44.911570 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":117,"threads_started":1}
I20260810 14:53:44.912673 7518 heartbeater.cc:347] Connected to a master server at 127.30.74.126:42265
I20260810 14:53:44.912791 7518 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:44.912981 7518 heartbeater.cc:511] Master 127.30.74.126:42265 requested a full tablet report, sending...
I20260810 14:53:44.913625 7317 ts_manager.cc:194] Registered new tserver with Master: 2d1f95c9eb00412aac7a87daf5d4de34 (127.30.74.65:39143)
I20260810 14:53:44.914224 7317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:35689
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:53:44.937489 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42265
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:39597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:45.020890 7523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:45.021112 7523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:45.021137 7523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:45.022382 7523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:45.022431 7523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:45.022459 7523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:53:45.023979 7523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:39597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:45.024302 7523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:45.024561 7523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:45.029589 7530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:45.029631 7533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:45.029630 7531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:45.029961 7523 server_base.cc:1034] running on GCE node
I20260810 14:53:45.030133 7523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:45.030341 7523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:45.031481 7523 hybrid_clock.cc:648] HybridClock initialized: now 1786373625031469 us; error 27 us; skew 500 ppm
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373625, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:45.034102 7523 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:53:45.034551 7523 webserver.cc:460] Webserver started at http://127.30.74.66:40123/ using document root <none> and password file <none>
I20260810 14:53:45.034822 7523 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:45.034895 7523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:45.035046 7523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:45.036129 7523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "50bb06599b2b45c4863aa058f85e4c53"
format_stamp: "Formatted at 2026-08-10 14:53:45 on dist-test-slave-0b58"
server_key: "4cd5a2be620cc5513a3c84e263c6de25"
server_key_iv: "e099ba4a55cfdf314ee4bd2d71c04214"
server_key_version: "encryptionkey@0"
I20260810 14:53:45.036562 7523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "50bb06599b2b45c4863aa058f85e4c53"
format_stamp: "Formatted at 2026-08-10 14:53:45 on dist-test-slave-0b58"
server_key: "4cd5a2be620cc5513a3c84e263c6de25"
server_key_iv: "e099ba4a55cfdf314ee4bd2d71c04214"
server_key_version: "encryptionkey@0"
I20260810 14:53:45.037922 7523 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:45.038672 7541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.038834 7523 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 14:53:45.038945 7523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "50bb06599b2b45c4863aa058f85e4c53"
format_stamp: "Formatted at 2026-08-10 14:53:45 on dist-test-slave-0b58"
server_key: "4cd5a2be620cc5513a3c84e263c6de25"
server_key_iv: "e099ba4a55cfdf314ee4bd2d71c04214"
server_key_version: "encryptionkey@0"
I20260810 14:53:45.039023 7523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:45.047679 7523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:45.049715 7523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:45.049880 7523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:45.050127 7523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:45.050484 7523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:45.050540 7523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.050587 7523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:45.050621 7523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.057255 7523 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:34069
I20260810 14:53:45.057348 7655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:34069 every 8 connection(s)
I20260810 14:53:45.057721 7523 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:53:45.063047 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 7523
I20260810 14:53:45.063179 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:53:45.063431 31017 external_mini_cluster.cc:1428] Setting key 66ff88944826ef7b1016aec849ecf40f
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373625, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:45.071182 7359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:45.058954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:37379 (local address 127.30.74.126:42265)
0810 14:53:45.059030 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:45.059033 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:45.059046 (+ 13us) server_negotiation.cc:407] Connection header received
0810 14:53:45.059792 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:45.059794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:45.059809 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:45.059877 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:45.061309 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.062236 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.063804 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.064021 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.064098 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.067258 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:45.067260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:45.067261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:45.067271 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:45.069293 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:45.069899 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:45.069900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:45.069901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:45.069969 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:45.070284 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:45.070285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:45.070285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:45.070434 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:45.070485 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:45.070952 (+ 467us) server_negotiation.cc:299] Negotiation successful
0810 14:53:45.071069 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 14:53:45.071913 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:45.058597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:37379)
0810 14:53:45.058812 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:45.058876 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:45.059637 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:45.059979 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:45.059983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:45.060883 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:45.061190 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.061198 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.062361 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.062362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.063720 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.063723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.064091 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.064093 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.064312 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.065339 (+ 1027us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:45.065349 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:45.067135 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:45.069413 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:45.069422 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:45.069425 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:45.069788 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:45.070054 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:45.070055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:45.070055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:45.070200 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:45.070541 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:45.070548 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:45.070777 (+ 229us) client_negotiation.cc:769] Sending connection context
0810 14:53:45.071399 (+ 622us) client_negotiation.cc:240] Negotiation successful
0810 14:53:45.071620 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":95,"threads_started":1}
I20260810 14:53:45.072748 7656 heartbeater.cc:347] Connected to a master server at 127.30.74.126:42265
I20260810 14:53:45.072860 7656 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:45.073066 7656 heartbeater.cc:511] Master 127.30.74.126:42265 requested a full tablet report, sending...
I20260810 14:53:45.073542 7317 ts_manager.cc:194] Registered new tserver with Master: 50bb06599b2b45c4863aa058f85e4c53 (127.30.74.66:34069)
I20260810 14:53:45.074092 7317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:37379
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:53:45.095484 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42265
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:39597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:45.183003 7661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:45.183235 7661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:45.183261 7661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:45.184509 7661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:45.184580 7661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:45.184608 7661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:53:45.186147 7661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:39597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:45.186399 7661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:45.186651 7661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:45.191573 7669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:45.191569 7666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:45.191567 7667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:45.192027 7661 server_base.cc:1034] running on GCE node
I20260810 14:53:45.192309 7661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:45.192535 7661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:45.193706 7661 hybrid_clock.cc:648] HybridClock initialized: now 1786373625193701 us; error 40 us; skew 500 ppm
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373625, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:45.196497 7661 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:53:45.197050 7661 webserver.cc:460] Webserver started at http://127.30.74.67:40663/ using document root <none> and password file <none>
I20260810 14:53:45.197492 7661 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:45.197547 7661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:45.197641 7661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:53:45.198779 7661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "9d8fe2fdde2345398fde84f73b0d4c09"
format_stamp: "Formatted at 2026-08-10 14:53:45 on dist-test-slave-0b58"
server_key: "2786a1210c88e736b4e314cef136ae0f"
server_key_iv: "1f9699db8a7ae4e450b99341b8d5ac29"
server_key_version: "encryptionkey@0"
I20260810 14:53:45.199092 7661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "9d8fe2fdde2345398fde84f73b0d4c09"
format_stamp: "Formatted at 2026-08-10 14:53:45 on dist-test-slave-0b58"
server_key: "2786a1210c88e736b4e314cef136ae0f"
server_key_iv: "1f9699db8a7ae4e450b99341b8d5ac29"
server_key_version: "encryptionkey@0"
I20260810 14:53:45.200431 7661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260810 14:53:45.201129 7676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.201344 7661 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.201436 7661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "9d8fe2fdde2345398fde84f73b0d4c09"
format_stamp: "Formatted at 2026-08-10 14:53:45 on dist-test-slave-0b58"
server_key: "2786a1210c88e736b4e314cef136ae0f"
server_key_iv: "1f9699db8a7ae4e450b99341b8d5ac29"
server_key_version: "encryptionkey@0"
I20260810 14:53:45.201509 7661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:45.239131 7661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:45.241333 7661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:45.241536 7661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:45.241817 7661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:45.242208 7661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:45.242244 7661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.242267 7661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:45.242311 7661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.248380 7661 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:44819
I20260810 14:53:45.248472 7798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:44819 every 8 connection(s)
I20260810 14:53:45.248811 7661 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:53:45.252806 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 7661
I20260810 14:53:45.252883 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:53:45.253049 31017 external_mini_cluster.cc:1428] Setting key 0dac8b0b26a2cd1c9ec93ee4db1c8425
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373625, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:53:45.258445 7359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:45.249582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:55497 (local address 127.30.74.126:42265)
0810 14:53:45.249657 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:45.249659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:45.249880 (+ 221us) server_negotiation.cc:407] Connection header received
0810 14:53:45.250427 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:45.250429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:45.250448 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:45.250545 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:45.251528 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.252053 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.253258 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.253459 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.253535 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.255654 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:45.255655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:45.255656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:45.255664 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:45.256953 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:45.257449 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:45.257450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:45.257451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:45.257512 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:45.257774 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:45.257775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:45.257775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:45.257899 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:45.257975 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:45.258274 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 14:53:45.258356 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 14:53:45.258631 7802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:45.249571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:55497)
0810 14:53:45.249754 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:45.249792 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:45.250312 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:45.250627 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:45.250631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:45.251215 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:45.251415 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.251419 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.252147 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.252148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.253157 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.253160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.253545 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.253546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.253683 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.254230 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:45.254238 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:45.255558 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:53:45.257078 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:45.257086 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:45.257088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:45.257347 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:45.257583 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:53:45.257584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:53:45.257584 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:53:45.257701 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:53:45.257978 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:53:45.257986 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:53:45.258130 (+ 144us) client_negotiation.cc:769] Sending connection context
0810 14:53:45.258235 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 14:53:45.258392 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260810 14:53:45.259146 7799 heartbeater.cc:347] Connected to a master server at 127.30.74.126:42265
I20260810 14:53:45.259243 7799 heartbeater.cc:464] Registering TS with master...
I20260810 14:53:45.259389 7799 heartbeater.cc:511] Master 127.30.74.126:42265 requested a full tablet report, sending...
I20260810 14:53:45.259752 7317 ts_manager.cc:194] Registered new tserver with Master: 9d8fe2fdde2345398fde84f73b0d4c09 (127.30.74.67:44819)
I20260810 14:53:45.260085 7317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:55497
I20260810 14:53:45.264631 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.65.keytab.
Entry for principal oryx/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.65.keytab.
WARNING: no policy specified for oryx/127.30.74.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.66.keytab.
Entry for principal oryx/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.66.keytab.
WARNING: no policy specified for oryx/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.67.keytab.
Entry for principal oryx/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.67.keytab.
I20260810 14:53:45.340392 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 7382
I20260810 14:53:45.347280 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 7523
I20260810 14:53:45.354748 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 7661
I20260810 14:53:45.361889 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 7283
I20260810 14:53:45.368204 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:42265
--webserver_interface=127.30.74.126
--webserver_port=44269
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:39597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:42265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:53:45.451841 7809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:45.452061 7809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:45.452085 7809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:45.453290 7809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:53:45.453336 7809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:45.453351 7809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:45.453363 7809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:53:45.453377 7809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:53:45.454824 7809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:39597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:42265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:42265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=44269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:45.455034 7809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:45.455209 7809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:45.460085 7814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:45.460125 7809 server_base.cc:1034] running on GCE node
W20260810 14:53:45.460098 7815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:45.460137 7817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:45.460579 7809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:45.460897 7809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:45.462067 7809 hybrid_clock.cc:648] HybridClock initialized: now 1786373625462043 us; error 39 us; skew 500 ppm
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373625, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:45.464723 7809 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:53:45.465200 7809 webserver.cc:460] Webserver started at http://127.30.74.126:44269/ using document root <none> and password file <none>
I20260810 14:53:45.465447 7809 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:45.465526 7809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:45.466938 7809 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 14:53:45.467633 7824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.467829 7809 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:45.467908 7809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "db2bccbc02bf41358d4d6da9df4dba62"
format_stamp: "Formatted at 2026-08-10 14:53:44 on dist-test-slave-0b58"
server_key: "57776d0dd624c16ce2aba54b1b41c9e3"
server_key_iv: "c0cf7ba0382e102fec9033170864ba37"
server_key_version: "encryptionkey@0"
I20260810 14:53:45.468350 7809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:45.484869 7809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:45.486836 7809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:45.486956 7809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:45.491079 7809 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:42265
I20260810 14:53:45.491115 7878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:42265 every 8 connection(s)
I20260810 14:53:45.491540 7809 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:53:45.493104 7880 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:53:45.493479 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 7809
I20260810 14:53:45.493774 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:39143
--local_ip_for_outbound_sockets=127.30.74.65
--tserver_master_addrs=127.30.74.126:42265
--webserver_port=40211
--webserver_interface=127.30.74.65
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:39597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.74.65
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.65.keytab with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:53:45.493788 7880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62: Bootstrap starting.
I20260810 14:53:45.501729 7880 log.cc:824] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62: Log is configured to *not* fsync() on all Append() calls
I20260810 14:53:45.503453 7880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:53:45.503764 7880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62: Bootstrap complete.
I20260810 14:53:45.506019 7880 raft_consensus.cc:348] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2bccbc02bf41358d4d6da9df4dba62" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42265 } }
I20260810 14:53:45.506296 7880 raft_consensus.cc:729] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2bccbc02bf41358d4d6da9df4dba62, State: Initialized, Role: FOLLOWER
I20260810 14:53:45.506505 7880 consensus_queue.cc:260] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2bccbc02bf41358d4d6da9df4dba62" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42265 } }
I20260810 14:53:45.506603 7880 raft_consensus.cc:388] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:53:45.506659 7880 raft_consensus.cc:482] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:53:45.506706 7880 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:53:45.507680 7880 raft_consensus.cc:504] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2bccbc02bf41358d4d6da9df4dba62" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42265 } }
I20260810 14:53:45.507812 7880 leader_election.cc:302] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db2bccbc02bf41358d4d6da9df4dba62; no voters:
I20260810 14:53:45.508006 7880 leader_election.cc:288] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 14:53:45.508074 7884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 14:53:45.508240 7884 raft_consensus.cc:686] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [term 2 LEADER]: Becoming Leader. State: Replica: db2bccbc02bf41358d4d6da9df4dba62, State: Running, Role: LEADER
I20260810 14:53:45.508306 7880 sys_catalog.cc:564] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:53:45.508356 7884 consensus_queue.cc:237] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2bccbc02bf41358d4d6da9df4dba62" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42265 } }
I20260810 14:53:45.508605 7884 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db2bccbc02bf41358d4d6da9df4dba62. Latest consensus state: current_term: 2 leader_uuid: "db2bccbc02bf41358d4d6da9df4dba62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2bccbc02bf41358d4d6da9df4dba62" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42265 } } }
I20260810 14:53:45.508677 7884 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:45.508607 7885 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "db2bccbc02bf41358d4d6da9df4dba62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2bccbc02bf41358d4d6da9df4dba62" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42265 } } }
I20260810 14:53:45.508874 7885 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62 [sys.catalog]: This master's current role is: LEADER
I20260810 14:53:45.509253 7889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:53:45.509832 7889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:53:45.509999 7889 catalog_manager.cc:1259] Loaded cluster ID: 095798fd05594c3c83719050aff84d7b
I20260810 14:53:45.510221 7889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:53:45.511034 7889 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:53:45.511229 7889 catalog_manager.cc:5879] T 00000000000000000000000000000000 P db2bccbc02bf41358d4d6da9df4dba62: Loaded TSK: 0
I20260810 14:53:45.511451 7889 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 14:53:45.590214 7882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:45.590467 7882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:45.590512 7882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:45.591697 7882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:45.591763 7882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:45.591812 7882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:53:45.593357 7882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:39597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.65.keytab
--principal=oryx/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:39143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=40211
--tserver_master_addrs=127.30.74.126:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:45.593616 7882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:45.593855 7882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:45.598954 7907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:45.599054 7906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:45.599242 7882 server_base.cc:1034] running on GCE node
W20260810 14:53:45.599059 7909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:45.599578 7882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:45.599896 7882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:45.601092 7882 hybrid_clock.cc:648] HybridClock initialized: now 1786373625601078 us; error 30 us; skew 500 ppm
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373625, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:45.604219 7882 init.cc:377] Logged in from keytab as oryx/127.30.74.65@KRBTEST.COM (short username oryx)
I20260810 14:53:45.604812 7882 webserver.cc:460] Webserver started at http://127.30.74.65:40211/ using document root <none> and password file <none>
I20260810 14:53:45.605075 7882 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:45.605165 7882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:45.607050 7882 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:53:45.607795 7916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.607973 7882 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:45.608073 7882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "2d1f95c9eb00412aac7a87daf5d4de34"
format_stamp: "Formatted at 2026-08-10 14:53:44 on dist-test-slave-0b58"
server_key: "8f4cbb539b8348af26d58c8ed0ee7533"
server_key_iv: "b692a0463b7b5408de9bfadadfb8c824"
server_key_version: "encryptionkey@0"
I20260810 14:53:45.608496 7882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:45.623896 7882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:45.626266 7882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:45.626413 7882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:45.626667 7882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:45.626988 7882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:45.627024 7882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.627048 7882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:45.627060 7882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.633829 7882 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:39143
I20260810 14:53:45.633877 8036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:39143 every 8 connection(s)
I20260810 14:53:45.634224 7882 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:53:45.640022 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 7882
I20260810 14:53:45.640456 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:34069
--local_ip_for_outbound_sockets=127.30.74.66
--tserver_master_addrs=127.30.74.126:42265
--webserver_port=40123
--webserver_interface=127.30.74.66
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:39597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.74.66
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.66.keytab with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:45.644652 8039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:45.635041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:55809)
0810 14:53:45.635226 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:45.635270 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:45.635738 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:45.636304 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:45.636308 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:45.636902 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:45.637096 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.637101 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.638813 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.638814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.639624 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.639627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.640479 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.640480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.640930 (+ 450us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.641570 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:45.641581 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:45.644408 (+ 2827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":100,"threads_started":1}
W20260810 14:53:45.644809 8040 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:45.635130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:55809 (local address 127.30.74.126:42265)
0810 14:53:45.635352 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:45.635361 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:45.635398 (+ 37us) server_negotiation.cc:407] Connection header received
0810 14:53:45.635960 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:45.635968 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:45.636028 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:45.636188 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:45.637243 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.638696 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.639736 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.640360 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.640484 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.644607 (+ 4123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:55809: BlockingRecv error: recv got EOF from 127.30.74.65:55809 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":104,"threads_started":1}
W20260810 14:53:45.645033 8037 heartbeater.cc:644] Failed to heartbeat to 127.30.74.126:42265 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.74.126:42265: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:45.651609 8039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:45.645352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:46217)
0810 14:53:45.645735 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:45.645746 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:45.645800 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:45.645966 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:45.645967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:45.646138 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:45.646195 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.646197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.647216 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.647217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.647976 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.647979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.648563 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.648564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.648665 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.649232 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:45.649238 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:45.651537 (+ 2299us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367}
W20260810 14:53:45.651750 8040 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:45.645422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:46217 (local address 127.30.74.126:42265)
0810 14:53:45.645705 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:45.645707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:45.645811 (+ 104us) server_negotiation.cc:407] Connection header received
0810 14:53:45.645867 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:45.645869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:45.645885 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:45.645951 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:45.646282 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.647116 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.648078 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.648472 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.648557 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.651668 (+ 3111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:46217: BlockingRecv error: recv got EOF from 127.30.74.65:46217 (error 108)
Metrics: {"server-negotiator.queue_time_us":262}
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:45.658706 8039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:45.652376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:55625)
0810 14:53:45.652468 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:45.652479 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:45.652515 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:45.652654 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:45.652655 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:45.652800 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:45.652835 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.652837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.653779 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.653780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.654573 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.654575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.655046 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.655047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.655101 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.655570 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:45.655575 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:45.658633 (+ 3058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":76}
W20260810 14:53:45.658802 8040 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:45.652428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:55625 (local address 127.30.74.126:42265)
0810 14:53:45.652494 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:45.652497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:45.652525 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:53:45.652578 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:45.652580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:45.652593 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:45.652652 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:45.652910 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.653696 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.654660 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.654970 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.655037 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.658733 (+ 3696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:55625: BlockingRecv error: recv got EOF from 127.30.74.65:55625 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 14:53:45.659072 8037 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:53:45.730068 8041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:45.730312 8041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:45.730358 8041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:45.731575 8041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:45.731644 8041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:45.731688 8041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:53:45.733247 8041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:39597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.66.keytab
--principal=oryx/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:34069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=40123
--tserver_master_addrs=127.30.74.126:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:45.733543 8041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:45.733788 8041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:45.739081 8047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:45.739086 8050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:45.739068 8048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:45.739420 8041 server_base.cc:1034] running on GCE node
I20260810 14:53:45.739643 8041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:45.739902 8041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:45.741070 8041 hybrid_clock.cc:648] HybridClock initialized: now 1786373625741055 us; error 45 us; skew 500 ppm
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373625, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:45.743578 8041 init.cc:377] Logged in from keytab as oryx/127.30.74.66@KRBTEST.COM (short username oryx)
I20260810 14:53:45.744051 8041 webserver.cc:460] Webserver started at http://127.30.74.66:40123/ using document root <none> and password file <none>
I20260810 14:53:45.744320 8041 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:45.744382 8041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:45.745715 8041 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:45.746320 8057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.746493 8041 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:53:45.746579 8041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "50bb06599b2b45c4863aa058f85e4c53"
format_stamp: "Formatted at 2026-08-10 14:53:45 on dist-test-slave-0b58"
server_key: "4cd5a2be620cc5513a3c84e263c6de25"
server_key_iv: "e099ba4a55cfdf314ee4bd2d71c04214"
server_key_version: "encryptionkey@0"
I20260810 14:53:45.746932 8041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:45.773825 8041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:45.775923 8041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:45.776053 8041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:45.776316 8041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:45.776625 8041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:45.776657 8041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.776679 8041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:45.776692 8041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.782976 8041 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:34069
I20260810 14:53:45.783063 8173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:34069 every 8 connection(s)
I20260810 14:53:45.783432 8041 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:53:45.787779 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 8041
I20260810 14:53:45.788132 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:44819
--local_ip_for_outbound_sockets=127.30.74.67
--tserver_master_addrs=127.30.74.126:42265
--webserver_port=40663
--webserver_interface=127.30.74.67
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:39597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.74.67
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.67.keytab with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 14:53:45.791782 8040 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:45.784303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:58071 (local address 127.30.74.126:42265)
0810 14:53:45.784362 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:45.784365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:45.784608 (+ 243us) server_negotiation.cc:407] Connection header received
0810 14:53:45.785121 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:45.785122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:45.785136 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:45.785192 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:45.786165 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.786658 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.787748 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.788014 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.788078 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.791721 (+ 3643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:58071: BlockingRecv error: recv got EOF from 127.30.74.66:58071 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 14:53:45.791875 8177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:45.784308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:58071)
0810 14:53:45.784473 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:45.784511 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:45.785026 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:45.785245 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:45.785248 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:45.785832 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:45.786030 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.786035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.786757 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.786757 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.787640 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.787643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.788101 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.788102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.788300 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.788850 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:45.788861 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:45.791624 (+ 2763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
W20260810 14:53:45.792120 8175 heartbeater.cc:644] Failed to heartbeat to 127.30.74.126:42265 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.74.126:42265: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:45.797060 8179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:45.792368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:39555)
0810 14:53:45.792533 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:45.792543 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:45.792635 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:45.792777 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:45.792778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:45.792965 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:45.793010 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.793012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.793665 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.793666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.794208 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.794210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.794615 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.794617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.794677 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.795200 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:45.795204 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:45.796957 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
W20260810 14:53:45.797099 8040 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:45.792402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:39555 (local address 127.30.74.126:42265)
0810 14:53:45.792453 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:45.792455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:45.792619 (+ 164us) server_negotiation.cc:407] Connection header received
0810 14:53:45.792699 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:45.792700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:45.792710 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:45.792767 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:45.793100 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.793574 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.794298 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.794506 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.794593 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.797035 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:39555: BlockingRecv error: recv got EOF from 127.30.74.66:39555 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:45.801884 8179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:45.797549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:52551)
0810 14:53:45.797597 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:45.797608 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:45.797652 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:45.797832 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:45.797832 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:45.797969 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:45.797999 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.798000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.798636 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.798637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.799142 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.799144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.799522 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.799523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.799583 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.800067 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:45.800071 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:45.801796 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 14:53:45.801939 8040 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:45.797640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:52551 (local address 127.30.74.126:42265)
0810 14:53:45.797709 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:45.797711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:45.797719 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:53:45.797739 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:45.797740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:45.797750 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:45.797826 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:45.798086 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.798550 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.799228 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.799436 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.799510 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.801869 (+ 2359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:52551: BlockingRecv error: recv got EOF from 127.30.74.66:52551 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260810 14:53:45.802168 8175 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:53:45.876359 8178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:53:45.876569 8178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:53:45.876591 8178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:53:45.877828 8178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:53:45.877878 8178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:53:45.877904 8178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:53:45.879364 8178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:39597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/krb5kdc/oryx_127.30.74.67.keytab
--principal=oryx/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:44819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=40663
--tserver_master_addrs=127.30.74.126:42265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:53:45.879618 8178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:53:45.879863 8178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:53:45.885022 8187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:45.885016 8184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:53:45.885016 8185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:53:45.885386 8178 server_base.cc:1034] running on GCE node
I20260810 14:53:45.885622 8178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:53:45.885874 8178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:53:45.887029 8178 hybrid_clock.cc:648] HybridClock initialized: now 1786373625887008 us; error 41 us; skew 500 ppm
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373625, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:53:45.889775 8178 init.cc:377] Logged in from keytab as oryx/127.30.74.67@KRBTEST.COM (short username oryx)
I20260810 14:53:45.890282 8178 webserver.cc:460] Webserver started at http://127.30.74.67:40663/ using document root <none> and password file <none>
I20260810 14:53:45.890514 8178 fs_manager.cc:359] Metadata directory not provided
I20260810 14:53:45.890592 8178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:53:45.891979 8178 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:53:45.892681 8195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.892858 8178 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.892964 8178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "9d8fe2fdde2345398fde84f73b0d4c09"
format_stamp: "Formatted at 2026-08-10 14:53:45 on dist-test-slave-0b58"
server_key: "2786a1210c88e736b4e314cef136ae0f"
server_key_iv: "1f9699db8a7ae4e450b99341b8d5ac29"
server_key_version: "encryptionkey@0"
I20260810 14:53:45.893359 8178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:53:45.903048 8178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:53:45.905162 8178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:53:45.905339 8178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:53:45.905593 8178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:53:45.905917 8178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:53:45.905974 8178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.906009 8178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:53:45.906055 8178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:53:45.912670 8178 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:44819
I20260810 14:53:45.912724 8318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:44819 every 8 connection(s)
I20260810 14:53:45.913087 8178 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:53:45.914880 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 8178
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
W20260810 14:53:45.922425 8040 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:45.913835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:42311 (local address 127.30.74.126:42265)
0810 14:53:45.913892 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:45.913895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:45.914167 (+ 272us) server_negotiation.cc:407] Connection header received
0810 14:53:45.914698 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:45.914699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:45.914709 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:45.914756 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:45.916138 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.916738 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.918111 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.918422 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.918501 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.922350 (+ 3849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:42311: BlockingRecv error: recv got EOF from 127.30.74.67:42311 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:53:45.922461 8321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:45.913840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:42311)
0810 14:53:45.914040 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:45.914077 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:45.914588 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:45.914849 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:45.914854 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:45.915664 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:45.916002 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.916009 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.916839 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.916840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.917997 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.918000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.918517 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.918518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.918686 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.919339 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:45.919350 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:45.922242 (+ 2892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
W20260810 14:53:45.922828 8319 heartbeater.cc:644] Failed to heartbeat to 127.30.74.126:42265 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.74.126:42265: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
I20260810 14:53:45.923944 8323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:45.915452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47782 (local address 127.30.74.126:42265)
0810 14:53:45.915579 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:45.915580 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:45.915690 (+ 110us) server_negotiation.cc:407] Connection header received
0810 14:53:45.915763 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:45.915764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:45.915773 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:45.915827 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:45.916617 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.917152 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.918227 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.918513 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.918579 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.920139 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:53:45.920144 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:53:45.920144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:53:45.920156 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:53:45.922237 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:45.922663 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:45.922667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:45.922667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:45.922797 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:53:45.923038 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:53:45.923038 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:53:45.923039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:53:45.923360 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:53:45.923467 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 14:53:45.923727 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 14:53:45.923846 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:45.927990 8321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:45.923089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:42663)
0810 14:53:45.923143 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:45.923154 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:45.923194 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:45.923325 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:45.923326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:45.923490 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:45.923548 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.923550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.924479 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.924480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.925185 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.925188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.925504 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.925505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.925559 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.926000 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:45.926006 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:45.927923 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 14:53:45.928467 8040 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:45.923152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:42663 (local address 127.30.74.126:42265)
0810 14:53:45.923193 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:45.923196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:45.923205 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:45.923247 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:45.923248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:45.923258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:45.923313 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:45.923647 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.924397 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.925258 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.925438 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.925491 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.928415 (+ 2924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:42663: BlockingRecv error: recv got EOF from 127.30.74.67:42663 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:45 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:45.933642 8321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:45.929536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:50527)
0810 14:53:45.929630 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:45.929644 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:45.929706 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:45.929828 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:45.929829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:45.929991 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:45.930020 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.930022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.930606 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.930607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.931082 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:45.931083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.931427 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.931428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:45.931475 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.931836 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:45.931840 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:45.933594 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":74}
W20260810 14:53:45.933750 8040 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:45.929582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:50527 (local address 127.30.74.126:42265)
0810 14:53:45.929629 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:45.929631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:45.929689 (+ 58us) server_negotiation.cc:407] Connection header received
0810 14:53:45.929764 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:45.929765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:45.929774 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:45.929825 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:45.930088 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.930537 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.931157 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:45.931337 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:45.931393 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:45.933682 (+ 2289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:50527: BlockingRecv error: recv got EOF from 127.30.74.67:50527 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 14:53:45.933804 8319 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 14:53:46 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:46 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:46.664219 8324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:46.659487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:50655)
0810 14:53:46.659610 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:46.659623 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:46.659696 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:46.659859 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:46.659860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:46.660016 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:46.660065 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:46.660066 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:46.660709 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:46.660710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:46.661248 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:46.661249 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:46.661596 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:46.661596 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:46.661637 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:46.662134 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:46.662139 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:46.664109 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
W20260810 14:53:46.664295 8325 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:46.659558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:50655 (local address 127.30.74.126:42265)
0810 14:53:46.659732 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:46.659734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:46.659743 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:46.659766 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:46.659767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:46.659777 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:46.659839 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:46.660133 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:46.660643 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:46.661342 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:46.661530 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:46.661583 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:46.664197 (+ 2614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:50655: BlockingRecv error: recv got EOF from 127.30.74.65:50655 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":84,"threads_started":1}
Aug 10 14:53:46 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:46 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:46.807579 8326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:46.802547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:34531)
0810 14:53:46.802722 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:46.802735 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:46.802843 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:46.803032 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:46.803033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:46.803212 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:46.803295 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:46.803297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:46.803971 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:46.803972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:46.804554 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:46.804556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:46.804933 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:46.804934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:46.805017 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:46.805542 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:46.805547 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:46.807480 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
W20260810 14:53:46.807631 8325 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:46.802644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:34531 (local address 127.30.74.126:42265)
0810 14:53:46.802724 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:46.802727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:46.802829 (+ 102us) server_negotiation.cc:407] Connection header received
0810 14:53:46.802926 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:46.802927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:46.802941 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:46.803009 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:46.803385 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:46.803884 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:46.804647 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:46.804847 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:46.804932 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:46.807564 (+ 2632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:34531: BlockingRecv error: recv got EOF from 127.30.74.66:34531 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 10 14:53:46 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:46 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:46.939332 8327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:46.934259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:45505)
0810 14:53:46.934419 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:46.934430 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:46.934488 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:46.934714 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:46.934715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:46.934885 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:46.934971 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:46.934974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:46.935904 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:46.935905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:46.936495 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:46.936498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:46.936897 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:46.936898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:46.936967 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:46.937391 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:46.937396 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:46.939262 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
W20260810 14:53:46.939404 8325 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:46.934335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:45505 (local address 127.30.74.126:42265)
0810 14:53:46.934391 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:46.934392 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:46.934568 (+ 176us) server_negotiation.cc:407] Connection header received
0810 14:53:46.934599 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:46.934600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:46.934614 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:46.934678 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:46.935072 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:46.935831 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:46.936590 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:46.936806 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:46.936894 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:46.939354 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:45505: BlockingRecv error: recv got EOF from 127.30.74.67:45505 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 10 14:53:47 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:47 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:47.669831 8328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:47.664873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:48711)
0810 14:53:47.665004 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:47.665014 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:47.665069 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:47.665297 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:47.665299 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:47.665468 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:47.665518 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:47.665520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:47.666191 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:47.666191 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:47.666716 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:47.666717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:47.667101 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:47.667102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:47.667144 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:47.667687 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:47.667692 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:47.669725 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
W20260810 14:53:47.670037 8329 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:47.664971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:48711 (local address 127.30.74.126:42265)
0810 14:53:47.665132 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:47.665133 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:47.665143 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:47.665168 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:47.665168 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:47.665178 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:47.665249 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:47.665597 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:47.666117 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:47.666821 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:47.667028 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:47.667093 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:47.669878 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:48711: BlockingRecv error: recv got EOF from 127.30.74.65:48711 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
Aug 10 14:53:47 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:47 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:47.813272 8330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:47.808356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:46981)
0810 14:53:47.808528 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:47.808538 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:47.808603 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:47.808733 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:47.808734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:47.808894 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:47.808944 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:47.808946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:47.809654 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:47.809655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:47.810177 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:47.810179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:47.810513 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:47.810514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:47.810553 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:47.811102 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:47.811106 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:47.813211 (+ 2105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":107,"threads_started":1}
W20260810 14:53:47.813396 8329 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:47.808423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:46981 (local address 127.30.74.126:42265)
0810 14:53:47.808503 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:47.808506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:47.808597 (+ 91us) server_negotiation.cc:407] Connection header received
0810 14:53:47.808651 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:47.808651 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:47.808663 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:47.808718 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:47.809080 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:47.809588 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:47.810268 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:47.810449 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:47.810499 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:47.813315 (+ 2816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:46981: BlockingRecv error: recv got EOF from 127.30.74.66:46981 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 14:53:47 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:47 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:47.945065 8332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:47.939842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:51895)
0810 14:53:47.940008 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:47.940021 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:47.940162 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:47.940377 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:47.940379 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:47.940564 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:47.940618 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:47.940621 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:47.941311 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:47.941312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:47.942073 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:47.942077 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:47.942461 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:47.942462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:47.942521 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:47.942988 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:47.942992 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:47.944996 (+ 2004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":63,"threads_started":1}
W20260810 14:53:47.945257 8329 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:47.940031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:51895 (local address 127.30.74.126:42265)
0810 14:53:47.940096 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:47.940098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:47.940108 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:47.940240 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:47.940241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:47.940297 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:47.940368 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:47.940717 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:47.941215 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:47.942158 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:47.942352 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:47.942461 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:47.945174 (+ 2713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:51895: BlockingRecv error: recv got EOF from 127.30.74.67:51895 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 10 14:53:48 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:48 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:48.675309 8333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:48.670413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:40905)
0810 14:53:48.670575 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:48.670585 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:48.670671 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:48.670836 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:48.670837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:48.671005 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:48.671057 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:48.671059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:48.672040 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:48.672041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:48.672628 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:48.672631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:48.672968 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:48.672968 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:48.673018 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:48.673471 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:48.673476 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:48.675232 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
W20260810 14:53:48.675459 8334 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:48.670480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:40905 (local address 127.30.74.126:42265)
0810 14:53:48.670661 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:48.670663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:48.670676 (+ 13us) server_negotiation.cc:407] Connection header received
0810 14:53:48.670744 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:48.670745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:48.670757 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:48.670813 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:48.671207 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:48.671968 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:48.672707 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:48.672897 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:48.672960 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:48.675340 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:40905: BlockingRecv error: recv got EOF from 127.30.74.65:40905 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
Aug 10 14:53:48 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:48 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:48.818784 8335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:48.813846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:59503)
0810 14:53:48.814034 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:48.814046 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:48.814119 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:48.814264 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:48.814265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:48.814437 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:48.814488 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:48.814490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:48.815249 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:48.815250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:48.815916 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:48.815919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:48.816323 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:48.816324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:48.816379 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:48.816905 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:48.816910 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:48.818725 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":76,"threads_started":1}
W20260810 14:53:48.818939 8334 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:48.813964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:59503 (local address 127.30.74.126:42265)
0810 14:53:48.814037 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:48.814039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:48.814114 (+ 75us) server_negotiation.cc:407] Connection header received
0810 14:53:48.814177 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:48.814178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:48.814190 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:48.814240 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:48.814552 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:48.815169 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:48.815997 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:48.816191 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:48.816248 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:48.818857 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:59503: BlockingRecv error: recv got EOF from 127.30.74.66:59503 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 14:53:48 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:48 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:48.951319 8336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:48.945689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:56245)
0810 14:53:48.945893 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:48.945905 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:48.946035 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:48.946281 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:48.946283 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:48.946543 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:48.946624 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:48.946645 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:48.947376 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:48.947377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:48.947951 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:48.947953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:48.948431 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:48.948432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:48.948497 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:48.949002 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:48.949007 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:48.951240 (+ 2233us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":113,"threads_started":1}
W20260810 14:53:48.951512 8334 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:48.945807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:56245 (local address 127.30.74.126:42265)
0810 14:53:48.945887 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:48.945890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:48.946109 (+ 219us) server_negotiation.cc:407] Connection header received
0810 14:53:48.946146 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:48.946163 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:48.946183 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:48.946243 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:48.946729 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:48.947266 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:48.948068 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:48.948349 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:48.948426 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:48.951434 (+ 3008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:56245: BlockingRecv error: recv got EOF from 127.30.74.67:56245 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 10 14:53:49 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:49 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:49.685078 8337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:49.675824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:45275)
0810 14:53:49.675970 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:49.675982 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:49.676042 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:49.676201 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:49.676202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:49.676439 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:49.676491 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:49.676492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:49.677133 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:49.677134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:49.677642 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:49.677643 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:49.677967 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:49.677968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:49.678008 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:49.682912 (+ 4904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:49.682919 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:49.685016 (+ 2097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
W20260810 14:53:49.685281 8339 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:49.675902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:45275 (local address 127.30.74.126:42265)
0810 14:53:49.676057 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:49.676059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:49.676069 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:49.676100 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:49.676100 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:49.676111 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:49.676183 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:49.676565 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:49.677067 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:49.677714 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:49.677901 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:49.677952 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:49.685163 (+ 7211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:45275: BlockingRecv error: recv got EOF from 127.30.74.65:45275 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
Aug 10 14:53:49 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:49 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:49.823918 8340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:49.819324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:39863)
0810 14:53:49.819508 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:49.819519 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:49.819587 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:49.819708 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:49.819709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:49.819878 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:49.819930 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:49.819931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:49.820585 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:49.820585 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:49.821144 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:49.821146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:49.821519 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:49.821520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:49.821572 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:49.822039 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:49.822043 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:49.823857 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":122,"threads_started":1}
W20260810 14:53:49.824055 8339 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:49.819383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:39863 (local address 127.30.74.126:42265)
0810 14:53:49.819457 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:49.819460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:49.819586 (+ 126us) server_negotiation.cc:407] Connection header received
0810 14:53:49.819638 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:49.819639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:49.819650 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:49.819696 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:49.819988 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:49.820515 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:49.821223 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:49.821426 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:49.821481 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:49.823971 (+ 2490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:39863: BlockingRecv error: recv got EOF from 127.30.74.66:39863 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 10 14:53:49 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:49 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:49.956967 8342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:49.951898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:47789)
0810 14:53:49.952049 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:49.952061 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:49.952134 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:49.952338 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:49.952339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:49.952507 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:49.952569 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:49.952570 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:49.953285 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:49.953286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:49.953851 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:49.953854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:49.954208 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:49.954209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:49.954251 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:49.954792 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:49.954797 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:49.956908 (+ 2111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
W20260810 14:53:49.957059 8339 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:49.951973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:47789 (local address 127.30.74.126:42265)
0810 14:53:49.952052 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:49.952055 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:49.952137 (+ 82us) server_negotiation.cc:407] Connection header received
0810 14:53:49.952201 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:49.952201 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:49.952216 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:49.952281 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:49.952663 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:49.953168 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:49.953944 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:49.954133 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:49.954194 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:49.956992 (+ 2798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:47789: BlockingRecv error: recv got EOF from 127.30.74.67:47789 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 10 14:53:50 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:50 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:50.690634 8343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:50.685656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:37093)
0810 14:53:50.685916 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:50.685927 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:50.685982 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:50.686176 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:50.686177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:50.686347 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:50.686399 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:50.686400 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:50.687097 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:50.687104 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:50.687673 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:50.687675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:50.688119 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:50.688120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:50.688180 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:50.688743 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:50.688749 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:50.690564 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":190,"threads_started":1}
W20260810 14:53:50.690940 8344 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:50.685745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:37093 (local address 127.30.74.126:42265)
0810 14:53:50.685926 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:50.685964 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:50.685980 (+ 16us) server_negotiation.cc:407] Connection header received
0810 14:53:50.686060 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:50.686061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:50.686072 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:50.686141 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:50.686474 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:50.686990 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:50.687777 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:50.687978 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:50.688040 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:50.690710 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:37093: BlockingRecv error: recv got EOF from 127.30.74.65:37093 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":220,"threads_started":1}
Aug 10 14:53:50 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:50 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:50.829162 8345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:50.824491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:56817)
0810 14:53:50.824644 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:50.824663 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:50.824728 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:50.824883 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:50.824884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:50.825063 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:50.825125 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:50.825127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:50.825789 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:50.825790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:50.826425 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:50.826428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:50.826769 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:50.826770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:50.826819 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:50.827273 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:50.827278 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:50.829092 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
W20260810 14:53:50.829286 8344 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:50.824622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:56817 (local address 127.30.74.126:42265)
0810 14:53:50.824665 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:50.824667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:50.824708 (+ 41us) server_negotiation.cc:407] Connection header received
0810 14:53:50.824802 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:50.824803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:50.824813 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:50.824855 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:50.825197 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:50.825687 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:50.826506 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:50.826688 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:50.826739 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:50.829193 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:56817: BlockingRecv error: recv got EOF from 127.30.74.66:56817 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 10 14:53:50 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:50 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:50.962466 8347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:50.957525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:44195)
0810 14:53:50.957760 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:50.957773 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:50.957864 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:50.958028 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:50.958029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:50.958218 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:50.958284 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:50.958286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:50.958984 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:50.958985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:50.959517 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:50.959519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:50.959881 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:50.959881 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:50.959940 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:50.960472 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:50.960477 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:50.962369 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":144,"threads_started":1}
W20260810 14:53:50.962527 8344 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:50.957633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:44195 (local address 127.30.74.126:42265)
0810 14:53:50.957728 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:50.957731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:50.957843 (+ 112us) server_negotiation.cc:407] Connection header received
0810 14:53:50.957928 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:50.957929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:50.957946 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:50.958019 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:50.958376 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:50.958889 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:50.959605 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:50.959804 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:50.959882 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:50.962463 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:44195: BlockingRecv error: recv got EOF from 127.30.74.67:44195 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 14:53:51 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:51 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:51.696174 8348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:51.691260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:36041)
0810 14:53:51.691421 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:51.691432 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:51.691490 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:51.691689 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:51.691690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:51.691858 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:51.691907 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:51.691908 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:51.692699 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:51.692700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:51.693262 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:51.693264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:51.693664 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:51.693665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:51.693716 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:51.694273 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:51.694277 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:51.696114 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
W20260810 14:53:51.696360 8349 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:51.691416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:36041 (local address 127.30.74.126:42265)
0810 14:53:51.691533 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:51.691535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:51.691544 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:51.691567 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:51.691568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:51.691581 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:51.691669 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:51.692006 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:51.692602 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:51.693358 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:51.693570 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:51.693639 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:51.696231 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:36041: BlockingRecv error: recv got EOF from 127.30.74.65:36041 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":63,"threads_started":1}
Aug 10 14:53:51 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:51 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:51.834795 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:51.829715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:47843)
0810 14:53:51.829932 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:51.829945 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:51.830047 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:51.830237 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:51.830238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:51.830436 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:51.830487 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:51.830489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:51.831177 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:51.831178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:51.831944 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:51.831948 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:51.832405 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:51.832405 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:51.832470 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:51.832926 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:51.832931 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:51.834727 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":116,"threads_started":1}
W20260810 14:53:51.834926 8349 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:51.829835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:47843 (local address 127.30.74.126:42265)
0810 14:53:51.829930 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:51.829932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:51.830023 (+ 91us) server_negotiation.cc:407] Connection header received
0810 14:53:51.830127 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:51.830128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:51.830146 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:51.830229 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:51.830577 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:51.831091 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:51.832023 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:51.832313 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:51.832402 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:51.834841 (+ 2439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:47843: BlockingRecv error: recv got EOF from 127.30.74.66:47843 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 14:53:51 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:51 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:51.967792 8351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:51.963126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:36829)
0810 14:53:51.963287 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:51.963298 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:51.963372 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:51.963554 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:51.963555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:51.963782 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:51.963844 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:51.963847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:51.964499 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:51.964500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:51.965027 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:51.965028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:51.965372 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:51.965372 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:51.965412 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:51.965883 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:51.965888 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:51.967732 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
W20260810 14:53:51.967922 8349 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:51.963232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:36829 (local address 127.30.74.126:42265)
0810 14:53:51.963318 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:51.963321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:51.963360 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:53:51.963456 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:51.963457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:51.963471 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:51.963520 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:51.963915 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:51.964439 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:51.965105 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:51.965305 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:51.965360 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:51.967839 (+ 2479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:36829: BlockingRecv error: recv got EOF from 127.30.74.67:36829 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 14:53:52 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:52 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:52.701802 8352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:52.696821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:54293)
0810 14:53:52.696972 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:52.696990 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:52.697089 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:52.697343 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:52.697344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:52.697586 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:52.697654 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:52.697656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:52.698500 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:52.698501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:52.699086 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:52.699088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:52.699477 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:52.699478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:52.699518 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:52.699977 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:52.699981 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:52.701738 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
W20260810 14:53:52.702018 8353 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:52.696943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:54293 (local address 127.30.74.126:42265)
0810 14:53:52.697115 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:52.697117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:52.697135 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:53:52.697205 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:52.697206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:52.697220 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:52.697306 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:52.697758 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:52.698427 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:52.699182 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:52.699406 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:52.699470 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:52.701953 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:54293: BlockingRecv error: recv got EOF from 127.30.74.65:54293 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
Aug 10 14:53:52 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:52 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:52.840360 8354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:52.835321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:46383)
0810 14:53:52.835512 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:52.835533 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:52.835611 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:52.835874 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:52.835876 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:52.836056 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:52.836108 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:52.836110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:52.836906 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:52.836907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:52.837488 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:52.837490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:52.837834 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:52.837835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:52.837878 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:52.838428 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:52.838432 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:52.840221 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":77,"threads_started":1}
W20260810 14:53:52.840461 8353 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:52.835470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:46383 (local address 127.30.74.126:42265)
0810 14:53:52.835557 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:52.835560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:52.835590 (+ 30us) server_negotiation.cc:407] Connection header received
0810 14:53:52.835733 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:52.835734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:52.835748 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:52.835812 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:52.836289 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:52.836813 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:52.837570 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:52.837767 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:52.837822 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:52.840372 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:46383: BlockingRecv error: recv got EOF from 127.30.74.66:46383 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 14:53:52 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:52 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:52.973011 8355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:52.968315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:46199)
0810 14:53:52.968441 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:52.968458 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:52.968517 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:52.968648 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:52.968648 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:52.968830 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:52.968880 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:52.968882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:52.969557 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:52.969558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:52.970089 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:52.970091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:52.970449 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:52.970450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:52.970506 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:52.971082 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:52.971087 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:52.972909 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
W20260810 14:53:52.973062 8353 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:52.968335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:46199 (local address 127.30.74.126:42265)
0810 14:53:52.968399 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:52.968401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:52.968512 (+ 111us) server_negotiation.cc:407] Connection header received
0810 14:53:52.968562 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:52.968562 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:52.968576 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:52.968633 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:52.968937 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:52.969485 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:52.970158 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:52.970358 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:52.970411 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:52.972994 (+ 2583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:46199: BlockingRecv error: recv got EOF from 127.30.74.67:46199 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 10 14:53:53 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:53 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:53.707340 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:53.702439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:33293)
0810 14:53:53.702726 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:53.702741 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:53.702823 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:53.703067 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:53.703068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:53.703241 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:53.703298 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:53.703300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:53.703950 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:53.703951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:53.704566 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:53.704569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:53.704906 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:53.704906 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:53.704946 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:53.705422 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:53.705426 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:53.707238 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":92,"threads_started":1}
W20260810 14:53:53.707402 8357 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:53.702720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:33293 (local address 127.30.74.126:42265)
0810 14:53:53.702901 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:53.702903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:53.702914 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:53:53.702962 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:53.702963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:53.702974 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:53.703046 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:53.703370 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:53.703878 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:53.704644 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:53.704847 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:53.704898 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:53.707327 (+ 2429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:33293: BlockingRecv error: recv got EOF from 127.30.74.65:33293 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":91,"threads_started":1}
Aug 10 14:53:53 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:53 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:53.845883 8358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:53.840884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:42909)
0810 14:53:53.841071 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:53.841085 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:53.841191 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:53.841349 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:53.841351 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:53.841537 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:53.841587 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:53.841589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:53.842411 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:53.842413 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:53.842974 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:53.842976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:53.843361 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:53.843362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:53.843429 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:53.843934 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:53.843939 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:53.845777 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
W20260810 14:53:53.845939 8357 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:53.841007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:42909 (local address 127.30.74.126:42265)
0810 14:53:53.841094 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:53.841097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:53.841169 (+ 72us) server_negotiation.cc:407] Connection header received
0810 14:53:53.841255 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:53.841256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:53.841270 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:53.841341 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:53.841671 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:53.842315 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:53.843062 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:53.843271 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:53.843359 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:53.845875 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:42909: BlockingRecv error: recv got EOF from 127.30.74.66:42909 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 14:53:53 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:53 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:53.978894 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:53.973540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:41077)
0810 14:53:53.973698 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:53.973714 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:53.973776 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:53.973922 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:53.973923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:53.974097 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:53.974151 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:53.974154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:53.974910 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:53.974911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:53.975463 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:53.975465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:53.975859 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:53.975860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:53.975912 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:53.976487 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:53.976491 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:53.978825 (+ 2334us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
W20260810 14:53:53.979017 8357 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:53.973618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:41077 (local address 127.30.74.126:42265)
0810 14:53:53.973689 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:53.973691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:53.973774 (+ 83us) server_negotiation.cc:407] Connection header received
0810 14:53:53.973839 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:53.973840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:53.973854 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:53.973901 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:53.974261 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:53.974793 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:53.975567 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:53.975766 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:53.975831 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:53.978940 (+ 3109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:41077: BlockingRecv error: recv got EOF from 127.30.74.67:41077 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 14:53:54 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:54 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:54.715780 8361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:54.707883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:36491)
0810 14:53:54.708062 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:54.708073 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:54.708144 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:54.708316 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:54.708316 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:54.708517 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:54.708570 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:54.708572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:54.709280 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:54.709281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:54.709889 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:54.709891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:54.710211 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:54.710212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:54.710256 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:54.713758 (+ 3502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:54.713763 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:54.715718 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":102,"threads_started":1}
W20260810 14:53:54.715953 8363 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:54.707980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:36491 (local address 127.30.74.126:42265)
0810 14:53:54.708093 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:54.708095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:54.708126 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:53:54.708201 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:54.708201 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:54.708213 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:54.708283 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:54.708630 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:54.709212 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:54.709954 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:54.710145 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:54.710194 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:54.715850 (+ 5656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:36491: BlockingRecv error: recv got EOF from 127.30.74.65:36491 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":57,"threads_started":1}
Aug 10 14:53:54 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:54 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:54.851588 8364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:54.846504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:55971)
0810 14:53:54.846686 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:54.846701 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:54.846765 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:54.846904 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:54.846905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:54.847077 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:54.847128 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:54.847129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:54.847756 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:54.847757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:54.848661 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:54.848664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:54.849041 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:54.849043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:54.849099 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:54.849546 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:54.849551 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:54.851519 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":113,"threads_started":1}
W20260810 14:53:54.851708 8363 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:54.846621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:55971 (local address 127.30.74.126:42265)
0810 14:53:54.846692 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:54.846694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:54.846767 (+ 73us) server_negotiation.cc:407] Connection header received
0810 14:53:54.846815 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:54.846815 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:54.846826 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:54.846887 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:54.847189 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:54.847678 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:54.848733 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:54.848934 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:54.849041 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:54.851612 (+ 2571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:55971: BlockingRecv error: recv got EOF from 127.30.74.66:55971 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 10 14:53:54 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:54 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:54.989445 8365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:54.979531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:53289)
0810 14:53:54.979682 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:54.979695 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:54.979764 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:54.979906 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:54.979907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:54.980081 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:54.980133 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:54.980135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:54.980837 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:54.980838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:54.981395 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:54.981396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:54.981713 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:54.981714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:54.981754 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:54.987269 (+ 5515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:54.987275 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:54.989348 (+ 2073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
W20260810 14:53:54.989586 8363 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:54.979584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:53289 (local address 127.30.74.126:42265)
0810 14:53:54.979657 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:54.979659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:54.979786 (+ 127us) server_negotiation.cc:407] Connection header received
0810 14:53:54.979814 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:54.979815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:54.979829 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:54.979892 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:54.980228 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:54.980755 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:54.981466 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:54.981647 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:54.981702 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:54.989502 (+ 7800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:53289: BlockingRecv error: recv got EOF from 127.30.74.67:53289 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 14:53:55 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:55 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:55.721148 8366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:55.716341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:54245)
0810 14:53:55.716515 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:55.716529 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:55.716598 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:55.716757 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:55.716758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:55.716938 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:55.716998 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:55.716999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:55.717881 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:55.717882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:55.718425 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:55.718427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:55.718740 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:55.718741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:55.718781 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:55.719234 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:55.719238 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:55.721077 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":74,"threads_started":1}
W20260810 14:53:55.721266 8367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:55.716403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:54245 (local address 127.30.74.126:42265)
0810 14:53:55.716573 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:55.716575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:55.716585 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:55.716661 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:55.716661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:55.716673 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:55.716736 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:55.717072 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:55.717815 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:55.718493 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:55.718682 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:55.718730 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:55.721174 (+ 2444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:54245: BlockingRecv error: recv got EOF from 127.30.74.65:54245 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":100,"threads_started":1}
Aug 10 14:53:55 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:55 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:55.856873 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:55.852074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:57189)
0810 14:53:55.852214 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:55.852225 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:55.852326 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:55.852476 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:55.852478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:55.852662 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:55.852712 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:55.852714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:55.853395 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:55.853396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:55.853988 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:55.853991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:55.854460 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:55.854461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:55.854505 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:55.855012 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:55.855017 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:55.856811 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
W20260810 14:53:55.857041 8367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:55.852216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:57189 (local address 127.30.74.126:42265)
0810 14:53:55.852333 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:55.852336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:55.852362 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:53:55.852396 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:55.852397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:55.852409 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:55.852467 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:55.852799 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:55.853310 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:55.854104 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:55.854383 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:55.854450 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:55.856951 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:57189: BlockingRecv error: recv got EOF from 127.30.74.66:57189 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 10 14:53:55 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:55 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:55.994828 8369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:55.990146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:48605)
0810 14:53:55.990291 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:55.990305 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:55.990375 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:55.990601 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:55.990602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:55.990848 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:55.990910 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:55.990913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:55.991587 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:55.991587 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:55.992111 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:55.992113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:55.992497 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:55.992497 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:55.992538 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:55.992969 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:55.992973 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:55.994766 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
W20260810 14:53:55.994973 8367 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:55.990270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:48605 (local address 127.30.74.126:42265)
0810 14:53:55.990315 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:55.990318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:55.990357 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:53:55.990472 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:55.990474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:55.990490 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:55.990559 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:55.990981 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:55.991511 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:55.992181 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:55.992429 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:55.992479 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:55.994882 (+ 2403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:48605: BlockingRecv error: recv got EOF from 127.30.74.67:48605 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 10 14:53:56 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:56 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:56.726586 8370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:56.721784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:52291)
0810 14:53:56.721929 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:56.721945 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:56.722003 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:56.722167 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:56.722168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:56.722356 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:56.722404 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:56.722405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:56.723055 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:56.723056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:56.723591 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:56.723592 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:56.723979 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:56.723980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:56.724020 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:56.724526 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:56.724532 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:56.726516 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
W20260810 14:53:56.726691 8371 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:56.721857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:52291 (local address 127.30.74.126:42265)
0810 14:53:56.722011 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:56.722018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:56.722053 (+ 35us) server_negotiation.cc:407] Connection header received
0810 14:53:56.722078 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:56.722079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:56.722091 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:56.722155 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:56.722476 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:56.722982 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:56.723703 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:56.723910 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:56.723972 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:56.726618 (+ 2646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:52291: BlockingRecv error: recv got EOF from 127.30.74.65:52291 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
Aug 10 14:53:56 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:56 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:56.861822 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:56.857324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:49287)
0810 14:53:56.857454 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:56.857470 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:56.857549 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:56.857672 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:56.857672 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:56.857849 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:56.857900 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:56.857902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:56.858560 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:56.858561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:56.859091 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:56.859093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:56.859465 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:56.859466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:56.859521 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:56.859983 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:56.859988 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:56.861762 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
W20260810 14:53:56.861963 8371 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:56.857388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:49287 (local address 127.30.74.126:42265)
0810 14:53:56.857437 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:56.857439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:56.857519 (+ 80us) server_negotiation.cc:407] Connection header received
0810 14:53:56.857608 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:56.857608 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:56.857619 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:56.857661 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:56.857963 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:56.858464 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:56.859203 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:56.859386 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:56.859438 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:56.861879 (+ 2441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:49287: BlockingRecv error: recv got EOF from 127.30.74.66:49287 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 10 14:53:56 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:56 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:57.000741 8373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:56.995357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:46831)
0810 14:53:56.995562 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:56.995576 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:56.995673 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:56.995862 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:56.995864 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:56.996064 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:56.996137 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:56.996140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:56.997011 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:56.997012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:56.997602 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:56.997604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:56.997943 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:56.997943 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:56.998001 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:56.998484 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:56.998489 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:57.000662 (+ 2173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":88,"threads_started":1}
W20260810 14:53:57.000923 8371 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:56.995461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:46831 (local address 127.30.74.126:42265)
0810 14:53:56.995547 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:56.995550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:56.995649 (+ 99us) server_negotiation.cc:407] Connection header received
0810 14:53:56.995752 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:56.995753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:56.995771 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:56.995854 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:56.996250 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:56.996943 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:56.997679 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:56.997874 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:56.997928 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:57.000773 (+ 2845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:46831: BlockingRecv error: recv got EOF from 127.30.74.67:46831 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 14:53:57 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:57 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:57.731812 8374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:57.727116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:37969)
0810 14:53:57.727264 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:57.727274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:57.727324 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:57.727513 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:57.727514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:57.727698 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:57.727752 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:57.727753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:57.728455 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:57.728456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:57.729002 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:57.729004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:57.729377 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:57.729378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:57.729419 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:57.729903 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:57.729908 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:57.731749 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
W20260810 14:53:57.731954 8375 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:57.727217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:37969 (local address 127.30.74.126:42265)
0810 14:53:57.727348 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:57.727350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:57.727359 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:53:57.727382 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:57.727383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:57.727394 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:57.727462 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:57.727825 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:57.728370 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:57.729096 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:57.729307 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:57.729369 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:57.731865 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:37969: BlockingRecv error: recv got EOF from 127.30.74.65:37969 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
Aug 10 14:53:57 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:57 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:57.867563 8376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:57.862366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:46907)
0810 14:53:57.862515 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:57.862532 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:57.862588 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:57.862752 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:57.862753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:57.862928 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:57.862978 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:57.862979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:57.863710 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:57.863712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:57.864292 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:57.864295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:57.864719 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:57.864720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:57.864798 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:57.865458 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:57.865463 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:57.867466 (+ 2003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
W20260810 14:53:57.867730 8375 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:57.862467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:46907 (local address 127.30.74.126:42265)
0810 14:53:57.862554 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:57.862556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:57.862570 (+ 14us) server_negotiation.cc:407] Connection header received
0810 14:53:57.862656 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:57.862656 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:57.862668 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:57.862734 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:57.863087 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:57.863618 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:57.864387 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:57.864655 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:57.864741 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:57.867652 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:46907: BlockingRecv error: recv got EOF from 127.30.74.66:46907 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 14:53:58 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:58 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:58.005961 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:58.001237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:52739)
0810 14:53:58.001391 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:58.001402 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:58.001465 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:58.001607 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:58.001608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:58.001788 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:58.001837 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:58.001838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:58.002571 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:58.002572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:58.003130 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:58.003131 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:58.003488 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:58.003489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:58.003530 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:58.003989 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:58.003994 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:58.005846 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
W20260810 14:53:58.005999 8375 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:58.001363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:52739 (local address 127.30.74.126:42265)
0810 14:53:58.001414 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:58.001416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:58.001459 (+ 43us) server_negotiation.cc:407] Connection header received
0810 14:53:58.001530 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:58.001531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:58.001542 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:58.001587 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:58.001934 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:58.002482 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:58.003224 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:58.003415 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:58.003480 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:58.005930 (+ 2450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:52739: BlockingRecv error: recv got EOF from 127.30.74.67:52739 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 10 14:53:58 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:58 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:58.737160 8378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:58.732506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:36343)
0810 14:53:58.732634 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:58.732648 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:58.732718 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:58.732850 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:58.732851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:58.733019 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:58.733068 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:58.733070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:58.733739 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:58.733739 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:58.734287 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:58.734289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:58.734608 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:58.734609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:58.734647 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:58.735103 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:58.735107 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:58.737089 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
W20260810 14:53:58.737288 8379 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:58.732568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:36343 (local address 127.30.74.126:42265)
0810 14:53:58.732731 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:58.732733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:58.732741 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:53:58.732762 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:58.732763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:58.732773 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:58.732832 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:58.733172 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:58.733675 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:58.734358 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:58.734546 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:58.734596 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:58.737202 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:36343: BlockingRecv error: recv got EOF from 127.30.74.65:36343 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
Aug 10 14:53:58 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:58 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:58.873344 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:58.868123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:47037)
0810 14:53:58.868362 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:58.868373 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:58.868438 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:58.868575 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:58.868576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:58.868747 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:58.868814 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:58.868816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:58.869501 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:58.869502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:58.870075 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:58.870077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:58.870515 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:58.870516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:58.870592 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:58.871131 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:58.871137 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:58.873232 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":97,"threads_started":1}
W20260810 14:53:58.873396 8379 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:58.868313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:47037 (local address 127.30.74.126:42265)
0810 14:53:58.868380 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:58.868382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:58.868433 (+ 51us) server_negotiation.cc:407] Connection header received
0810 14:53:58.868503 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:58.868504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:58.868515 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:58.868558 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:58.868893 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:58.869421 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:58.870172 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:58.870431 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:58.870507 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:58.873328 (+ 2821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:47037: BlockingRecv error: recv got EOF from 127.30.74.66:47037 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 10 14:53:59 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:59 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:59.011708 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:59.006545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:59957)
0810 14:53:59.006748 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:59.006761 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:59.006859 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:59.007014 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:59.007015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:59.007208 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:59.007280 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:59.007282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:59.007972 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:59.007973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:59.008672 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:59.008675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:59.009065 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:59.009066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:59.009136 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:59.009718 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:59.009723 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:59.011607 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":118,"threads_started":1}
W20260810 14:53:59.011775 8379 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:59.006612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:59957 (local address 127.30.74.126:42265)
0810 14:53:59.006711 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:59.006713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:59.006836 (+ 123us) server_negotiation.cc:407] Connection header received
0810 14:53:59.006931 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:59.006932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:59.006948 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:59.006997 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:59.007362 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:59.007878 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:59.008776 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:59.008979 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:59.009062 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:59.011693 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:59957: BlockingRecv error: recv got EOF from 127.30.74.67:59957 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 14:53:59 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:59 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:59.752175 8383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:59.737660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:39029)
0810 14:53:59.737816 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:59.737827 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:59.737886 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:59.738030 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:59.738031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:59.738203 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:59.738252 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:59.738253 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:59.738901 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:59.738902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:59.739449 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:59.739451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:59.739762 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:59.739763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:59.739801 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:59.750037 (+ 10236us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:59.750042 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:59.752102 (+ 2060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
W20260810 14:53:59.752322 8384 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:59.737727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:39029 (local address 127.30.74.126:42265)
0810 14:53:59.737863 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:59.737865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:59.737875 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:53:59.737947 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:59.737947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:59.737959 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:59.738011 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:59.738317 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:59.738835 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:59.739517 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:59.739703 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:59.739752 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:59.752198 (+ 12446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:39029: BlockingRecv error: recv got EOF from 127.30.74.65:39029 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
Aug 10 14:53:59 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:53:59 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:53:59.878201 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:53:59.873815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:47465)
0810 14:53:59.873950 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 14:53:59.873961 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:53:59.874030 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:53:59.874159 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:53:59.874160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:53:59.874323 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:53:59.874374 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:59.874376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:59.874986 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:59.874986 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:59.875521 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:53:59.875522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:53:59.875844 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:53:59.875844 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:53:59.875882 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:59.876333 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:53:59.876337 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:53:59.878094 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
W20260810 14:53:59.878235 8384 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:53:59.873890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:47465 (local address 127.30.74.126:42265)
0810 14:53:59.873965 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 14:53:59.873967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:53:59.874012 (+ 45us) server_negotiation.cc:407] Connection header received
0810 14:53:59.874077 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:53:59.874078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:53:59.874089 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:53:59.874140 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:53:59.874432 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:59.874923 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:59.875587 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:53:59.875781 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:53:59.875834 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:53:59.878175 (+ 2341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:47465: BlockingRecv error: recv got EOF from 127.30.74.66:47465 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 10 14:54:00 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:54:00 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.67@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:54:00.017344 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:00.012408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.67:60835)
0810 14:54:00.012573 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:00.012588 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:00.012676 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:00.013003 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:00.013006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:00.013277 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:54:00.013342 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:00.013345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:00.014022 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:00.014023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:00.014556 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:00.014558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:00.014893 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:00.014894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:00.014935 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:00.015380 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:54:00.015384 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:54:00.017280 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
W20260810 14:54:00.017462 8384 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:54:00.012529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:60835 (local address 127.30.74.126:42265)
0810 14:54:00.012599 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:00.012602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:00.012660 (+ 58us) server_negotiation.cc:407] Connection header received
0810 14:54:00.012755 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:00.012756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:00.012773 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:00.012832 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:00.013433 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:00.013938 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:00.014634 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:00.014829 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:00.014884 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:00.017375 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.67:60835: BlockingRecv error: recv got EOF from 127.30.74.67:60835 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 10 14:54:00 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:54:00 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.65@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:54:00.758002 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:00.752786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.65:49067)
0810 14:54:00.752947 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:00.752964 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:00.753019 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:00.753197 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:00.753198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:00.753365 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:54:00.753416 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:00.753417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:00.754315 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:00.754316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:00.754959 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:00.754963 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:00.755409 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:00.755411 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:00.755458 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:00.756021 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:54:00.756026 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:54:00.757939 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
W20260810 14:54:00.758206 8388 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:54:00.752905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:49067 (local address 127.30.74.126:42265)
0810 14:54:00.753037 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:00.753039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:00.753049 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:54:00.753077 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:00.753078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:00.753088 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:00.753157 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:00.753531 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:00.754187 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:00.755083 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:00.755294 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:00.755395 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:00.758113 (+ 2718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.65:49067: BlockingRecv error: recv got EOF from 127.30.74.65:49067 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
Aug 10 14:54:00 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:54:00 dist-test-slave-0b58 krb5kdc[7269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.74.66@KRBTEST.COM for oryx/127.30.74.126@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:54:00.884117 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:00.878808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42265 (local address 127.30.74.66:46567)
0810 14:54:00.878987 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:00.878998 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:00.879056 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:00.879249 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:00.879249 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:00.879421 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:54:00.879483 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:00.879485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:00.880214 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:00.880216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:00.880851 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:00.880854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:00.881404 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:00.881405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:00.881482 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:00.882105 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:54:00.882110 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:54:00.884054 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.74.126:42265: Server oryx/127.30.74.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":98,"threads_started":1}
W20260810 14:54:00.884420 8388 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:54:00.878957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:46567 (local address 127.30.74.126:42265)
0810 14:54:00.879034 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:00.879035 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:00.879047 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:54:00.879150 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:00.879151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:00.879162 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:00.879230 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:00.879572 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:00.880082 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:00.880954 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:00.881260 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:00.881345 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:00.884308 (+ 2963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.74.66:46567: BlockingRecv error: recv got EOF from 127.30.74.66:46567 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 14:54:00.925563 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 7882
I20260810 14:54:00.933370 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 8041
I20260810 14:54:00.940196 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 8178
I20260810 14:54:00.949805 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 7809
2026-08-10T14:54:00Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17605 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T14:54:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:54:00Z Disabled control of system clock
I20260810 14:54:00.982424 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:36981
--webserver_interface=127.30.74.126
--webserver_port=0
--builtin_ntp_servers=127.30.74.84:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:36981
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 14:54:01.069927 8394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:54:01.070192 8394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:54:01.070217 8394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:54:01.071528 8394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:54:01.071575 8394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:54:01.071588 8394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:54:01.071601 8394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:54:01.071614 8394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:54:01.073061 8394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:36981
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:36981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:54:01.073292 8394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:54:01.073557 8394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:54:01.078756 8403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:01.078985 8400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:01.078994 8401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:01.079290 8394 server_base.cc:1034] running on GCE node
I20260810 14:54:01.079577 8394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:54:01.079865 8394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:54:01.081010 8394 hybrid_clock.cc:648] HybridClock initialized: now 1786373641080995 us; error 30 us; skew 500 ppm
I20260810 14:54:01.082381 8394 webserver.cc:460] Webserver started at http://127.30.74.126:33057/ using document root <none> and password file <none>
I20260810 14:54:01.082657 8394 fs_manager.cc:359] Metadata directory not provided
I20260810 14:54:01.082731 8394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:54:01.082875 8394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:54:01.084002 8394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "db46f2bd6d00402da5b97f988fb38e2e"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "019a9fb4e74a225b4ae2094c25718440"
server_key_iv: "29f89b95b55bb8c25ee575c64aeaa6aa"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.084398 8394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "db46f2bd6d00402da5b97f988fb38e2e"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "019a9fb4e74a225b4ae2094c25718440"
server_key_iv: "29f89b95b55bb8c25ee575c64aeaa6aa"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.085788 8394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:54:01.086566 8410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:01.086771 8394 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:54:01.086863 8394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "db46f2bd6d00402da5b97f988fb38e2e"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "019a9fb4e74a225b4ae2094c25718440"
server_key_iv: "29f89b95b55bb8c25ee575c64aeaa6aa"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.086947 8394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:54:01.096889 8394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:54:01.097191 8394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:54:01.097337 8394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:54:01.101320 8394 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:36981
I20260810 14:54:01.101395 8463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:36981 every 8 connection(s)
I20260810 14:54:01.101735 8394 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:54:01.102415 8464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:54:01.105108 8464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e: Bootstrap starting.
I20260810 14:54:01.105688 8464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e: Neither blocks nor log segments found. Creating new log.
I20260810 14:54:01.105922 8464 log.cc:824] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:54:01.106679 8464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e: No bootstrap required, opened a new log
I20260810 14:54:01.107580 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 8394
I20260810 14:54:01.107705 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:54:01.107925 31017 external_mini_cluster.cc:1428] Setting key 2bb0b59ecd60087160c823660f5bae6a
I20260810 14:54:01.108999 8464 raft_consensus.cc:348] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db46f2bd6d00402da5b97f988fb38e2e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 36981 } }
I20260810 14:54:01.109177 8464 raft_consensus.cc:374] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:54:01.109267 8464 raft_consensus.cc:729] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db46f2bd6d00402da5b97f988fb38e2e, State: Initialized, Role: FOLLOWER
I20260810 14:54:01.109414 8464 consensus_queue.cc:260] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db46f2bd6d00402da5b97f988fb38e2e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 36981 } }
I20260810 14:54:01.109496 8464 raft_consensus.cc:388] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:54:01.109519 8464 raft_consensus.cc:482] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:54:01.109587 8464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:54:01.110116 8464 raft_consensus.cc:504] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db46f2bd6d00402da5b97f988fb38e2e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 36981 } }
I20260810 14:54:01.110201 8464 leader_election.cc:302] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db46f2bd6d00402da5b97f988fb38e2e; no voters:
I20260810 14:54:01.110409 8464 leader_election.cc:288] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:54:01.110455 8470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:54:01.110605 8470 raft_consensus.cc:686] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [term 1 LEADER]: Becoming Leader. State: Replica: db46f2bd6d00402da5b97f988fb38e2e, State: Running, Role: LEADER
I20260810 14:54:01.110700 8464 sys_catalog.cc:564] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:54:01.110692 8470 consensus_queue.cc:237] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db46f2bd6d00402da5b97f988fb38e2e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 36981 } }
I20260810 14:54:01.112552 8472 sys_catalog.cc:455] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db46f2bd6d00402da5b97f988fb38e2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db46f2bd6d00402da5b97f988fb38e2e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 36981 } } }
I20260810 14:54:01.112682 8472 sys_catalog.cc:458] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [sys.catalog]: This master's current role is: LEADER
I20260810 14:54:01.112553 8471 sys_catalog.cc:455] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader db46f2bd6d00402da5b97f988fb38e2e. Latest consensus state: current_term: 1 leader_uuid: "db46f2bd6d00402da5b97f988fb38e2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db46f2bd6d00402da5b97f988fb38e2e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 36981 } } }
I20260810 14:54:01.112805 8471 sys_catalog.cc:458] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e [sys.catalog]: This master's current role is: LEADER
W20260810 14:54:01.112942 8485 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:54:01.113083 8485 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:54:01.113291 8487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:54:01.113699 8487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:54:01.114418 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:01.108479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41596 (local address 127.30.74.126:36981)
0810 14:54:01.108713 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:01.108721 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:01.108746 (+ 25us) server_negotiation.cc:407] Connection header received
0810 14:54:01.108956 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:01.108961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:01.109014 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:01.109159 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:01.109398 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:01.111017 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:01.111904 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:01.112580 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:01.112707 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:01.113669 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:01.113674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:01.113674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:54:01.113895 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:01.113938 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:54:01.113953 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:01.114026 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:01.114121 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 14:54:01.114202 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
I20260810 14:54:01.115317 8487 catalog_manager.cc:1347] Generated new cluster ID: 5b91bc4ec0254ce2a79902442fc8784c
I20260810 14:54:01.115377 8487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:54:01.159137 8487 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:54:01.159701 8487 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:54:01.167120 8487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db46f2bd6d00402da5b97f988fb38e2e: Generated new TSK 0
I20260810 14:54:01.167343 8487 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:54:01.174352 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:36981
--builtin_ntp_servers=127.30.74.84:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 14:54:01.266727 8491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:54:01.266945 8491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:54:01.266968 8491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:54:01.268083 8491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:54:01.268128 8491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:54:01.268152 8491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:54:01.269534 8491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:36981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:54:01.269780 8491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:54:01.270139 8491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:54:01.275542 8500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:01.275548 8496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:01.275673 8491 server_base.cc:1034] running on GCE node
W20260810 14:54:01.275573 8497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:01.275995 8491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:54:01.276304 8491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:54:01.277462 8491 hybrid_clock.cc:648] HybridClock initialized: now 1786373641277443 us; error 37 us; skew 500 ppm
I20260810 14:54:01.278880 8491 webserver.cc:460] Webserver started at http://127.30.74.65:37001/ using document root <none> and password file <none>
I20260810 14:54:01.279147 8491 fs_manager.cc:359] Metadata directory not provided
I20260810 14:54:01.279244 8491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:54:01.279394 8491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:54:01.280560 8491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "caed0ca9531849029b89398e4d96ea43"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "81e308d51c19055a2f2e57ceb53e5d10"
server_key_iv: "7418292dc7ae3f03768c344912901c45"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.280943 8491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "caed0ca9531849029b89398e4d96ea43"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "81e308d51c19055a2f2e57ceb53e5d10"
server_key_iv: "7418292dc7ae3f03768c344912901c45"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.282322 8491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:54:01.283191 8507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:01.283392 8491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:54:01.283486 8491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "caed0ca9531849029b89398e4d96ea43"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "81e308d51c19055a2f2e57ceb53e5d10"
server_key_iv: "7418292dc7ae3f03768c344912901c45"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.283566 8491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:54:01.304291 8491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:54:01.304644 8491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:54:01.304805 8491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:54:01.305063 8491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:54:01.305416 8491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:54:01.305469 8491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:01.305505 8491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:54:01.305549 8491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:01.311777 8491 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:41531
I20260810 14:54:01.311879 8625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:41531 every 8 connection(s)
I20260810 14:54:01.312244 8491 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:54:01.317795 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:01.313151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:46215 (local address 127.30.74.126:36981)
0810 14:54:01.313212 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:01.313214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:01.313535 (+ 321us) server_negotiation.cc:407] Connection header received
0810 14:54:01.314031 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:01.314032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:01.314044 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:01.314099 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:01.314791 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:01.315357 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:01.316395 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:01.316598 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:01.316657 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:01.317537 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:01.317539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:01.317539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:54:01.317544 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:01.317561 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:54:01.317565 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:01.317621 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:01.317697 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 14:54:01.317727 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 14:54:01.318030 8628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:01.313139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:36981 (local address 127.30.74.65:46215)
0810 14:54:01.313406 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:01.313442 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:01.313943 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:01.314156 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:01.314160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:01.314169 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:54:01.314683 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:01.314687 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:01.315449 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:01.315450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:01.316229 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:01.316231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:01.316675 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:01.316676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:01.316789 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:01.317374 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:54:01.317374 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:54:01.317404 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:54:01.317429 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:54:01.317629 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:54:01.317633 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:54:01.317635 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 14:54:01.317688 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 14:54:01.317770 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":143,"threads_started":1}
I20260810 14:54:01.318593 8626 heartbeater.cc:347] Connected to a master server at 127.30.74.126:36981
I20260810 14:54:01.318701 8626 heartbeater.cc:464] Registering TS with master...
I20260810 14:54:01.318892 8626 heartbeater.cc:511] Master 127.30.74.126:36981 requested a full tablet report, sending...
I20260810 14:54:01.319422 8428 ts_manager.cc:194] Registered new tserver with Master: caed0ca9531849029b89398e4d96ea43 (127.30.74.65:41531)
I20260810 14:54:01.320060 8428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.74.65:46215
I20260810 14:54:01.320803 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 8491
I20260810 14:54:01.320880 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:54:01.321082 31017 external_mini_cluster.cc:1428] Setting key abc922ff36332f7005047de49f14773a
I20260810 14:54:01.322238 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:36981
--builtin_ntp_servers=127.30.74.84:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 14:54:01.408783 8629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:54:01.409014 8629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:54:01.409039 8629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:54:01.410292 8629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:54:01.410342 8629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:54:01.410369 8629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:54:01.411962 8629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:36981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:54:01.412283 8629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:54:01.412672 8629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:54:01.418046 8634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:01.418083 8637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:01.418061 8635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:01.418350 8629 server_base.cc:1034] running on GCE node
I20260810 14:54:01.418597 8629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:54:01.418883 8629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:54:01.420053 8629 hybrid_clock.cc:648] HybridClock initialized: now 1786373641420041 us; error 33 us; skew 500 ppm
I20260810 14:54:01.421672 8629 webserver.cc:460] Webserver started at http://127.30.74.66:42059/ using document root <none> and password file <none>
I20260810 14:54:01.421985 8629 fs_manager.cc:359] Metadata directory not provided
I20260810 14:54:01.422063 8629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:54:01.422211 8629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:54:01.423360 8629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "41886ab78c644941b3debbea4acdef33"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "35fc53b1261a43b282ba386b1780dfd2"
server_key_iv: "7ec6735129e3f7e371ff158129db098f"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.423751 8629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "41886ab78c644941b3debbea4acdef33"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "35fc53b1261a43b282ba386b1780dfd2"
server_key_iv: "7ec6735129e3f7e371ff158129db098f"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.425269 8629 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:54:01.426263 8643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:01.426465 8629 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:54:01.426579 8629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "41886ab78c644941b3debbea4acdef33"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "35fc53b1261a43b282ba386b1780dfd2"
server_key_iv: "7ec6735129e3f7e371ff158129db098f"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.426666 8629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:54:01.437422 8629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:54:01.437789 8629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:54:01.437953 8629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:54:01.438203 8629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:54:01.438566 8629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:54:01.438617 8629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:01.438658 8629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:54:01.438691 8629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:01.444842 8629 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:44145
I20260810 14:54:01.444933 8763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:44145 every 8 connection(s)
I20260810 14:54:01.445281 8629 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:54:01.448122 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 8629
I20260810 14:54:01.448207 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:54:01.448448 31017 external_mini_cluster.cc:1428] Setting key 1fd6799b0c306998a89012413daaf5f8
I20260810 14:54:01.449525 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:36981
--builtin_ntp_servers=127.30.74.84:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 14:54:01.451157 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:01.446112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:36705 (local address 127.30.74.126:36981)
0810 14:54:01.446195 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:01.446197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:01.446418 (+ 221us) server_negotiation.cc:407] Connection header received
0810 14:54:01.446984 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:01.446986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:01.447000 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:01.447061 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:01.447807 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:01.448434 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:01.449635 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:01.449868 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:01.449950 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:01.450862 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:01.450864 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:01.450864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:54:01.450869 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:01.450885 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:54:01.450890 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:01.450953 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:01.451042 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 14:54:01.451076 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 14:54:01.451421 8766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:01.446022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:36981 (local address 127.30.74.66:36705)
0810 14:54:01.446285 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:01.446323 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:01.446867 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:01.447137 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:01.447141 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:01.447155 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:54:01.447702 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:01.447707 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:01.448534 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:01.448535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:01.449545 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:01.449548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:01.449956 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:01.449958 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:01.450097 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:01.450716 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:54:01.450717 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:54:01.450748 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:54:01.450771 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:54:01.450954 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:54:01.450960 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:54:01.450964 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:54:01.451044 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 14:54:01.451209 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260810 14:54:01.451987 8764 heartbeater.cc:347] Connected to a master server at 127.30.74.126:36981
I20260810 14:54:01.452099 8764 heartbeater.cc:464] Registering TS with master...
I20260810 14:54:01.452358 8764 heartbeater.cc:511] Master 127.30.74.126:36981 requested a full tablet report, sending...
I20260810 14:54:01.452769 8428 ts_manager.cc:194] Registered new tserver with Master: 41886ab78c644941b3debbea4acdef33 (127.30.74.66:44145)
I20260810 14:54:01.453317 8428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.74.66:36705
W20260810 14:54:01.537196 8767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:54:01.537422 8767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:54:01.537447 8767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:54:01.538556 8767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:54:01.538594 8767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:54:01.538618 8767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:54:01.539870 8767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:36981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:54:01.540063 8767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:54:01.540369 8767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 14:54:01.545563 8767 server_base.cc:1034] running on GCE node
W20260810 14:54:01.545567 8772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:01.545643 8773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:01.545740 8776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:01.545975 8767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:54:01.546245 8767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:54:01.547417 8767 hybrid_clock.cc:648] HybridClock initialized: now 1786373641547390 us; error 58 us; skew 500 ppm
I20260810 14:54:01.548816 8767 webserver.cc:460] Webserver started at http://127.30.74.67:44639/ using document root <none> and password file <none>
I20260810 14:54:01.549067 8767 fs_manager.cc:359] Metadata directory not provided
I20260810 14:54:01.549139 8767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:54:01.549312 8767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:54:01.550442 8767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "d0afa66b0be84eb5961de85a86b5ac63"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "666797c4576bf9de0f4a775aae94e18f"
server_key_iv: "bf1ab858219823e3ca5c25ec3e0ea462"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.550830 8767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "d0afa66b0be84eb5961de85a86b5ac63"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "666797c4576bf9de0f4a775aae94e18f"
server_key_iv: "bf1ab858219823e3ca5c25ec3e0ea462"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.552352 8767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260810 14:54:01.553220 8783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:01.553409 8767 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:54:01.553519 8767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "d0afa66b0be84eb5961de85a86b5ac63"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "666797c4576bf9de0f4a775aae94e18f"
server_key_iv: "bf1ab858219823e3ca5c25ec3e0ea462"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.553608 8767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:54:01.566903 8767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:54:01.567263 8767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:54:01.567420 8767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:54:01.567673 8767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:54:01.568043 8767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:54:01.568097 8767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:01.568142 8767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:54:01.568181 8767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:01.574640 8767 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:33443
I20260810 14:54:01.574698 8906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:33443 every 8 connection(s)
I20260810 14:54:01.575043 8767 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:54:01.576164 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 8767
I20260810 14:54:01.576339 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:54:01.576557 31017 external_mini_cluster.cc:1428] Setting key 4c4dbdee7d41d3f425605d7084becba5
I20260810 14:54:01.580610 8469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:01.575809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:45195 (local address 127.30.74.126:36981)
0810 14:54:01.575883 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:01.575885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:01.576132 (+ 247us) server_negotiation.cc:407] Connection header received
0810 14:54:01.576820 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:01.576822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:01.576837 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:01.576885 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:01.577565 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:01.578069 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:01.579121 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:01.579329 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:01.579406 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:01.580316 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:01.580317 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:01.580318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:54:01.580322 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:01.580340 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:54:01.580349 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:01.580409 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:01.580490 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 14:54:01.580524 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 14:54:01.580860 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:01.575761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:36981 (local address 127.30.74.67:45195)
0810 14:54:01.575981 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:01.576026 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:01.576699 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:01.576968 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:01.576973 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:01.576987 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:54:01.577475 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:01.577479 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:01.578181 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:01.578183 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:01.579011 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:01.579014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:01.579419 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:01.579420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:01.579569 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:01.580140 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:54:01.580141 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:54:01.580171 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:54:01.580197 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:54:01.580410 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:54:01.580417 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:54:01.580421 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:54:01.580484 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 14:54:01.580595 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":109,"threads_started":1}
I20260810 14:54:01.581363 8907 heartbeater.cc:347] Connected to a master server at 127.30.74.126:36981
I20260810 14:54:01.581486 8907 heartbeater.cc:464] Registering TS with master...
I20260810 14:54:01.581682 8907 heartbeater.cc:511] Master 127.30.74.126:36981 requested a full tablet report, sending...
I20260810 14:54:01.582182 8428 ts_manager.cc:194] Registered new tserver with Master: d0afa66b0be84eb5961de85a86b5ac63 (127.30.74.67:33443)
I20260810 14:54:01.582538 8428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.74.67:45195
I20260810 14:54:01.588245 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 14:54:01.590237 8469 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:54:01.589683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41606 (local address 127.30.74.126:36981)
0810 14:54:01.589742 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:01.589744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:01.589884 (+ 140us) server_negotiation.cc:407] Connection header received
0810 14:54:01.589953 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:01.589954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:01.589966 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:01.590023 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:01.590165 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41606: BlockingRecv error: recv got EOF from 127.0.0.1:41606 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 14:54:01.590447 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 8491
I20260810 14:54:01.597440 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 8629
I20260810 14:54:01.603462 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 8767
I20260810 14:54:01.611876 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 8394
2026-08-10T14:54:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (661 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T14:54:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:54:01Z Disabled control of system clock
I20260810 14:54:01.643280 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:41451
--webserver_interface=127.30.74.126
--webserver_port=0
--builtin_ntp_servers=127.30.74.84:39373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:41451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 14:54:01.730178 8920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:54:01.730403 8920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:54:01.730425 8920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:54:01.731534 8920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:54:01.731570 8920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:54:01.731582 8920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:54:01.731595 8920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:54:01.731607 8920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:54:01.732933 8920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:39373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:41451
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:41451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:54:01.733196 8920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:54:01.733497 8920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:54:01.738715 8926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:01.738686 8928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:01.738708 8925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:01.739061 8920 server_base.cc:1034] running on GCE node
I20260810 14:54:01.739336 8920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:54:01.739643 8920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:54:01.740856 8920 hybrid_clock.cc:648] HybridClock initialized: now 1786373641740853 us; error 42 us; skew 500 ppm
I20260810 14:54:01.742333 8920 webserver.cc:460] Webserver started at http://127.30.74.126:37509/ using document root <none> and password file <none>
I20260810 14:54:01.742611 8920 fs_manager.cc:359] Metadata directory not provided
I20260810 14:54:01.742686 8920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:54:01.742844 8920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:54:01.743937 8920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "4c418d5422b843bf9b03a7acce9d5ca1"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "2276cff7817071640c5a9602176a0550"
server_key_iv: "9af39c39bb8e8f933ecb1c2b41466843"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.744378 8920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "4c418d5422b843bf9b03a7acce9d5ca1"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "2276cff7817071640c5a9602176a0550"
server_key_iv: "9af39c39bb8e8f933ecb1c2b41466843"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.745713 8920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:54:01.746562 8934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:01.746747 8920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:54:01.746871 8920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "4c418d5422b843bf9b03a7acce9d5ca1"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "2276cff7817071640c5a9602176a0550"
server_key_iv: "9af39c39bb8e8f933ecb1c2b41466843"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.746950 8920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:54:01.750478 8920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:54:01.750764 8920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:54:01.750912 8920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:54:01.755442 8920 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:41451
I20260810 14:54:01.755489 8989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:41451 every 8 connection(s)
I20260810 14:54:01.755879 8920 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:54:01.756703 8990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:54:01.758412 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 8920
I20260810 14:54:01.758540 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:54:01.758750 31017 external_mini_cluster.cc:1428] Setting key 085ce5ddab5a5b4e2670bc283d402f7a
I20260810 14:54:01.759989 8990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1: Bootstrap starting.
I20260810 14:54:01.760795 8990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1: Neither blocks nor log segments found. Creating new log.
I20260810 14:54:01.761134 8990 log.cc:824] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1: Log is configured to *not* fsync() on all Append() calls
I20260810 14:54:01.761694 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:01.759336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60034 (local address 127.30.74.126:41451)
0810 14:54:01.759545 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:01.759554 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:01.759588 (+ 34us) server_negotiation.cc:407] Connection header received
0810 14:54:01.760123 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:01.760133 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:01.760188 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:01.760407 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:01.761112 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:01.761116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:01.761116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:54:01.761120 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:01.761168 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:54:01.761183 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:01.761301 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:01.761452 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 14:54:01.761521 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260810 14:54:01.761932 8990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1: No bootstrap required, opened a new log
I20260810 14:54:01.763767 8990 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c418d5422b843bf9b03a7acce9d5ca1" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41451 } }
I20260810 14:54:01.763962 8990 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:54:01.763989 8990 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c418d5422b843bf9b03a7acce9d5ca1, State: Initialized, Role: FOLLOWER
I20260810 14:54:01.764138 8990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c418d5422b843bf9b03a7acce9d5ca1" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41451 } }
I20260810 14:54:01.764221 8990 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:54:01.764245 8990 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:54:01.764331 8990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:54:01.765244 8990 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c418d5422b843bf9b03a7acce9d5ca1" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41451 } }
I20260810 14:54:01.765376 8990 leader_election.cc:302] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c418d5422b843bf9b03a7acce9d5ca1; no voters:
I20260810 14:54:01.765552 8990 leader_election.cc:288] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:54:01.765609 8995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:54:01.765753 8995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [term 1 LEADER]: Becoming Leader. State: Replica: 4c418d5422b843bf9b03a7acce9d5ca1, State: Running, Role: LEADER
I20260810 14:54:01.765856 8995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c418d5422b843bf9b03a7acce9d5ca1" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41451 } }
I20260810 14:54:01.765879 8990 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:54:01.766245 8996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c418d5422b843bf9b03a7acce9d5ca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c418d5422b843bf9b03a7acce9d5ca1" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41451 } } }
I20260810 14:54:01.766362 8996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [sys.catalog]: This master's current role is: LEADER
I20260810 14:54:01.766255 8998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c418d5422b843bf9b03a7acce9d5ca1. Latest consensus state: current_term: 1 leader_uuid: "4c418d5422b843bf9b03a7acce9d5ca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c418d5422b843bf9b03a7acce9d5ca1" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41451 } } }
I20260810 14:54:01.766633 8998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1 [sys.catalog]: This master's current role is: LEADER
W20260810 14:54:01.767930 9012 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:54:01.767992 9012 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:54:01.768043 9001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:54:01.768227 9001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:54:01.769888 9001 catalog_manager.cc:1347] Generated new cluster ID: 6f44f97b304a4d03865bfe4e4ade8309
I20260810 14:54:01.769958 9001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:54:01.796195 9001 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:54:01.796463 9001 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:54:01.804461 9001 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c418d5422b843bf9b03a7acce9d5ca1: Generated new TSK 0
I20260810 14:54:01.804720 9001 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:54:01.806263 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:41451
--builtin_ntp_servers=127.30.74.84:39373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 14:54:01.898370 9015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:54:01.898600 9015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:54:01.898624 9015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:54:01.899842 9015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:54:01.899889 9015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:54:01.899916 9015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:54:01.901355 9015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:39373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:41451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:54:01.901616 9015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:54:01.901978 9015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:54:01.907912 9020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:01.908021 9021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:01.908021 9023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:01.909022 9015 server_base.cc:1034] running on GCE node
I20260810 14:54:01.909329 9015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:54:01.909642 9015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:54:01.910836 9015 hybrid_clock.cc:648] HybridClock initialized: now 1786373641910812 us; error 43 us; skew 500 ppm
I20260810 14:54:01.912516 9015 webserver.cc:460] Webserver started at http://127.30.74.65:37231/ using document root <none> and password file <none>
I20260810 14:54:01.912791 9015 fs_manager.cc:359] Metadata directory not provided
I20260810 14:54:01.912869 9015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:54:01.913046 9015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:54:01.914216 9015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "8ee463d0754e4a36901d83ee436441c4"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "8ef5d65301ec4944ff501efa996dbf5f"
server_key_iv: "748939d3d2bb3c410d44c885c0352f8c"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.914628 9015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "8ee463d0754e4a36901d83ee436441c4"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "8ef5d65301ec4944ff501efa996dbf5f"
server_key_iv: "748939d3d2bb3c410d44c885c0352f8c"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.916092 9015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:54:01.917080 9032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:01.917296 9015 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:54:01.917426 9015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "8ee463d0754e4a36901d83ee436441c4"
format_stamp: "Formatted at 2026-08-10 14:54:01 on dist-test-slave-0b58"
server_key: "8ef5d65301ec4944ff501efa996dbf5f"
server_key_iv: "748939d3d2bb3c410d44c885c0352f8c"
server_key_version: "encryptionkey@0"
I20260810 14:54:01.917518 9015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:54:01.921176 9015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:54:01.921483 9015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:54:01.921635 9015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:54:01.921864 9015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:54:01.922189 9015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:54:01.922243 9015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:01.922286 9015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:54:01.922312 9015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:01.928376 9015 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:42191
I20260810 14:54:01.928454 9153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:42191 every 8 connection(s)
I20260810 14:54:01.928826 9015 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:54:01.932055 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:01.929637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:35773 (local address 127.30.74.126:41451)
0810 14:54:01.929685 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:01.929687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:01.929935 (+ 248us) server_negotiation.cc:407] Connection header received
0810 14:54:01.930445 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:01.930447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:01.930453 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:01.930523 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:01.931773 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:01.931774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:01.931775 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:54:01.931780 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:01.931796 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:54:01.931800 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:01.931855 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:01.931985 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 14:54:01.931999 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 14:54:01.932319 9156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:01.929599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:41451 (local address 127.30.74.65:35773)
0810 14:54:01.929795 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:01.929837 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:01.930308 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:01.930599 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:01.930603 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:01.930611 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:54:01.931593 (+ 982us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:54:01.931593 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:54:01.931624 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:54:01.931662 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:54:01.931892 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:54:01.931898 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:54:01.931901 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:54:01.931981 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 14:54:01.932035 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260810 14:54:01.933090 9154 heartbeater.cc:347] Connected to a master server at 127.30.74.126:41451
I20260810 14:54:01.933254 9154 heartbeater.cc:464] Registering TS with master...
I20260810 14:54:01.933413 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 9015
I20260810 14:54:01.933459 9154 heartbeater.cc:511] Master 127.30.74.126:41451 requested a full tablet report, sending...
I20260810 14:54:01.933501 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:54:01.933768 31017 external_mini_cluster.cc:1428] Setting key a4dffc792bc6636ed57a34d0b3479575
I20260810 14:54:01.934089 8951 ts_manager.cc:194] Registered new tserver with Master: 8ee463d0754e4a36901d83ee436441c4 (127.30.74.65:42191)
I20260810 14:54:01.935024 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:41451
--builtin_ntp_servers=127.30.74.84:39373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 14:54:02.019093 9157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:54:02.019363 9157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:54:02.019404 9157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:54:02.020740 9157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:54:02.020807 9157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:54:02.020855 9157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:54:02.022375 9157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:39373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:41451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:54:02.022673 9157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:54:02.023007 9157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:54:02.028326 9164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:02.028380 9163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:02.028326 9166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:02.028601 9157 server_base.cc:1034] running on GCE node
I20260810 14:54:02.028956 9157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:54:02.029276 9157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:54:02.030460 9157 hybrid_clock.cc:648] HybridClock initialized: now 1786373642030443 us; error 46 us; skew 500 ppm
I20260810 14:54:02.031898 9157 webserver.cc:460] Webserver started at http://127.30.74.66:38067/ using document root <none> and password file <none>
I20260810 14:54:02.032177 9157 fs_manager.cc:359] Metadata directory not provided
I20260810 14:54:02.032316 9157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:54:02.032493 9157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:54:02.033726 9157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "44b66fbb58bb442da22cb66aae9d7a24"
format_stamp: "Formatted at 2026-08-10 14:54:02 on dist-test-slave-0b58"
server_key: "f26ee5b509747753e7e524038f791db0"
server_key_iv: "e42a1c44112726040a6505dbc72dc42e"
server_key_version: "encryptionkey@0"
I20260810 14:54:02.034173 9157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "44b66fbb58bb442da22cb66aae9d7a24"
format_stamp: "Formatted at 2026-08-10 14:54:02 on dist-test-slave-0b58"
server_key: "f26ee5b509747753e7e524038f791db0"
server_key_iv: "e42a1c44112726040a6505dbc72dc42e"
server_key_version: "encryptionkey@0"
I20260810 14:54:02.035665 9157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:54:02.036496 9172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:02.036685 9157 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:54:02.036808 9157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "44b66fbb58bb442da22cb66aae9d7a24"
format_stamp: "Formatted at 2026-08-10 14:54:02 on dist-test-slave-0b58"
server_key: "f26ee5b509747753e7e524038f791db0"
server_key_iv: "e42a1c44112726040a6505dbc72dc42e"
server_key_version: "encryptionkey@0"
I20260810 14:54:02.036891 9157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:54:02.040796 9157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:54:02.041103 9157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:54:02.041273 9157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:54:02.041543 9157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:54:02.041903 9157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:54:02.041962 9157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:02.042006 9157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:54:02.042039 9157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:02.048839 9157 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:43173
I20260810 14:54:02.048890 9291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:43173 every 8 connection(s)
I20260810 14:54:02.049283 9157 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:54:02.050796 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 9157
I20260810 14:54:02.050913 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:54:02.051151 31017 external_mini_cluster.cc:1428] Setting key d844cf9f235e5d79cdcf0e29a553379a
I20260810 14:54:02.052240 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:41451
--builtin_ntp_servers=127.30.74.84:39373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 14:54:02.053102 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:02.050124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:58579 (local address 127.30.74.126:41451)
0810 14:54:02.050209 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:02.050212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:02.050433 (+ 221us) server_negotiation.cc:407] Connection header received
0810 14:54:02.051157 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:02.051158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:02.051165 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:02.051278 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:02.052779 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:02.052780 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:02.052781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:54:02.052787 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:02.052805 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:54:02.052811 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:02.052885 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:02.052969 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 14:54:02.052987 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 14:54:02.053210 9294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:02.050039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:41451 (local address 127.30.74.66:58579)
0810 14:54:02.050287 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:02.050332 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:02.051028 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:02.051386 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:02.051391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:02.051405 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:54:02.052616 (+ 1211us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:54:02.052617 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:54:02.052648 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:54:02.052679 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:54:02.052890 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:54:02.052894 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:54:02.052896 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 14:54:02.052957 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 14:54:02.053011 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":120,"threads_started":1}
I20260810 14:54:02.053730 9292 heartbeater.cc:347] Connected to a master server at 127.30.74.126:41451
I20260810 14:54:02.053843 9292 heartbeater.cc:464] Registering TS with master...
I20260810 14:54:02.054003 9292 heartbeater.cc:511] Master 127.30.74.126:41451 requested a full tablet report, sending...
I20260810 14:54:02.054409 8950 ts_manager.cc:194] Registered new tserver with Master: 44b66fbb58bb442da22cb66aae9d7a24 (127.30.74.66:43173)
W20260810 14:54:02.140780 9295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:54:02.141021 9295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:54:02.141048 9295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:54:02.142230 9295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:54:02.142269 9295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:54:02.142295 9295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:54:02.143649 9295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:39373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:41451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:54:02.143899 9295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:54:02.144202 9295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:54:02.149369 9301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:02.149384 9302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:02.149365 9304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:02.149621 9295 server_base.cc:1034] running on GCE node
I20260810 14:54:02.149857 9295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:54:02.150166 9295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:54:02.151365 9295 hybrid_clock.cc:648] HybridClock initialized: now 1786373642151334 us; error 38 us; skew 500 ppm
I20260810 14:54:02.152781 9295 webserver.cc:460] Webserver started at http://127.30.74.67:37911/ using document root <none> and password file <none>
I20260810 14:54:02.153062 9295 fs_manager.cc:359] Metadata directory not provided
I20260810 14:54:02.153137 9295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:54:02.153298 9295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:54:02.154407 9295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "d317f5777e7c4106aae88f3ab3c4b541"
format_stamp: "Formatted at 2026-08-10 14:54:02 on dist-test-slave-0b58"
server_key: "b4e62d88f9b3483b2749eefc23e1d707"
server_key_iv: "c339911c2189302a133cca172280ae84"
server_key_version: "encryptionkey@0"
I20260810 14:54:02.154785 9295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "d317f5777e7c4106aae88f3ab3c4b541"
format_stamp: "Formatted at 2026-08-10 14:54:02 on dist-test-slave-0b58"
server_key: "b4e62d88f9b3483b2749eefc23e1d707"
server_key_iv: "c339911c2189302a133cca172280ae84"
server_key_version: "encryptionkey@0"
I20260810 14:54:02.156150 9295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:54:02.157044 9310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:02.157258 9295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:54:02.157388 9295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "d317f5777e7c4106aae88f3ab3c4b541"
format_stamp: "Formatted at 2026-08-10 14:54:02 on dist-test-slave-0b58"
server_key: "b4e62d88f9b3483b2749eefc23e1d707"
server_key_iv: "c339911c2189302a133cca172280ae84"
server_key_version: "encryptionkey@0"
I20260810 14:54:02.157474 9295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:54:02.161017 9295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:54:02.161310 9295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:54:02.161459 9295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:54:02.161677 9295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:54:02.161994 9295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:54:02.162047 9295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:02.162088 9295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:54:02.162117 9295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:02.168946 9295 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:42277
I20260810 14:54:02.168987 9431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:42277 every 8 connection(s)
I20260810 14:54:02.169348 9295 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:54:02.172577 8993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:02.170103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:43079 (local address 127.30.74.126:41451)
0810 14:54:02.170152 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:02.170155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:02.170410 (+ 255us) server_negotiation.cc:407] Connection header received
0810 14:54:02.170872 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:02.170873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:02.170879 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:02.170953 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:02.172174 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:02.172175 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:02.172176 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:54:02.172181 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:02.172197 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:54:02.172203 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:02.172327 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:02.172507 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 14:54:02.172523 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:54:02.172719 9434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:02.170062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:41451 (local address 127.30.74.67:43079)
0810 14:54:02.170277 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:02.170318 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:02.170780 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:02.171007 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:02.171011 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:02.171019 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:54:02.172012 (+ 993us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:54:02.172012 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:54:02.172041 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:54:02.172071 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:54:02.172401 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:54:02.172406 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:54:02.172409 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:54:02.172478 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 14:54:02.172532 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":108,"threads_started":1}
I20260810 14:54:02.173228 9432 heartbeater.cc:347] Connected to a master server at 127.30.74.126:41451
I20260810 14:54:02.173386 9432 heartbeater.cc:464] Registering TS with master...
I20260810 14:54:02.173574 9432 heartbeater.cc:511] Master 127.30.74.126:41451 requested a full tablet report, sending...
I20260810 14:54:02.174029 8950 ts_manager.cc:194] Registered new tserver with Master: d317f5777e7c4106aae88f3ab3c4b541 (127.30.74.67:42277)
I20260810 14:54:02.178925 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 9295
I20260810 14:54:02.179064 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:54:02.179369 31017 external_mini_cluster.cc:1428] Setting key 9ecc07a2d39962110d63c4d609cbfd2d
I20260810 14:54:02.180531 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 14:54:02.182672 8993 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:54:02.182116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60042 (local address 127.30.74.126:41451)
0810 14:54:02.182196 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:02.182199 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:02.182329 (+ 130us) server_negotiation.cc:407] Connection header received
0810 14:54:02.182373 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:02.182373 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:02.182378 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:02.182431 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:02.182594 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60042: BlockingRecv error: recv got EOF from 127.0.0.1:60042 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 14:54:02.182942 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 9015
I20260810 14:54:02.193094 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 9157
I20260810 14:54:02.202806 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 9295
I20260810 14:54:02.211272 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 8920
2026-08-10T14:54:02Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (599 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:54:02 dist-test-slave-0b58 krb5kdc[9442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:54:02 dist-test-slave-0b58 krb5kdc[9442](info): set up 2 sockets
Aug 10 14:54:02 dist-test-slave-0b58 krb5kdc[9442](info): commencing operation
krb5kdc: starting...
W20260810 14:54:03.300127 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:54:03 dist-test-slave-0b58 krb5kdc[9442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:54:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:54:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:54:03.431692 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:44277
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:44277
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:54:03.519585 9455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:54:03.519835 9455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:54:03.519861 9455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:54:03.521173 9455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:54:03.521222 9455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:54:03.521237 9455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:54:03.521252 9455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:54:03.521265 9455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:54:03.522886 9455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:44277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:44277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:54:03.523195 9455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:54:03.523454 9455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:54:03.529134 9461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:03.529114 9462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:03.529210 9455 server_base.cc:1034] running on GCE node
W20260810 14:54:03.529114 9464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:03.529722 9455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:54:03.530035 9455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:54:03.531224 9455 hybrid_clock.cc:648] HybridClock initialized: now 1786373643531186 us; error 71 us; skew 500 ppm
Aug 10 14:54:03 dist-test-slave-0b58 krb5kdc[9442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373643, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:54:03.534133 9455 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:54:03.534576 9455 webserver.cc:460] Webserver started at http://127.30.74.126:36307/ using document root <none> and password file <none>
I20260810 14:54:03.534771 9455 fs_manager.cc:359] Metadata directory not provided
I20260810 14:54:03.534814 9455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:54:03.534910 9455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:54:03.536033 9455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "d64668f5d23248ecb467bcfa78ec1576"
format_stamp: "Formatted at 2026-08-10 14:54:03 on dist-test-slave-0b58"
server_key: "b3d7b771dcb9fc9ce1cd7c93119d5b23"
server_key_iv: "7a468c5baf258c89b259e9af7bd4a94e"
server_key_version: "encryptionkey@0"
I20260810 14:54:03.536469 9455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "d64668f5d23248ecb467bcfa78ec1576"
format_stamp: "Formatted at 2026-08-10 14:54:03 on dist-test-slave-0b58"
server_key: "b3d7b771dcb9fc9ce1cd7c93119d5b23"
server_key_iv: "7a468c5baf258c89b259e9af7bd4a94e"
server_key_version: "encryptionkey@0"
I20260810 14:54:03.537951 9455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:54:03.538789 9471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:03.539000 9455 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:54:03.539094 9455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "d64668f5d23248ecb467bcfa78ec1576"
format_stamp: "Formatted at 2026-08-10 14:54:03 on dist-test-slave-0b58"
server_key: "b3d7b771dcb9fc9ce1cd7c93119d5b23"
server_key_iv: "7a468c5baf258c89b259e9af7bd4a94e"
server_key_version: "encryptionkey@0"
I20260810 14:54:03.539155 9455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:54:03.550531 9455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:54:03.553059 9455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:54:03.553191 9455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:54:03.557242 9455 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:44277
I20260810 14:54:03.557283 9527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:44277 every 8 connection(s)
I20260810 14:54:03.557673 9455 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:54:03.557919 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 9455
I20260810 14:54:03.558035 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:54:03.558272 31017 external_mini_cluster.cc:1428] Setting key 99fd9d5bf693d6b6cbe756b93bb77109
I20260810 14:54:03.558952 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:54:03.562407 9529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576: Bootstrap starting.
I20260810 14:54:03.563113 9529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576: Neither blocks nor log segments found. Creating new log.
I20260810 14:54:03.563447 9529 log.cc:824] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576: Log is configured to *not* fsync() on all Append() calls
I20260810 14:54:03.564203 9529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576: No bootstrap required, opened a new log
Aug 10 14:54:03 dist-test-slave-0b58 krb5kdc[9442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:54:03.566480 9529 raft_consensus.cc:348] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64668f5d23248ecb467bcfa78ec1576" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44277 } }
I20260810 14:54:03.566623 9529 raft_consensus.cc:374] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:54:03.566671 9529 raft_consensus.cc:729] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d64668f5d23248ecb467bcfa78ec1576, State: Initialized, Role: FOLLOWER
I20260810 14:54:03.566823 9529 consensus_queue.cc:260] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64668f5d23248ecb467bcfa78ec1576" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44277 } }
I20260810 14:54:03.566920 9529 raft_consensus.cc:388] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:54:03.566967 9529 raft_consensus.cc:482] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:54:03.567039 9529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:54:03.567606 9529 raft_consensus.cc:504] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64668f5d23248ecb467bcfa78ec1576" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44277 } }
I20260810 14:54:03.567734 9529 leader_election.cc:302] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d64668f5d23248ecb467bcfa78ec1576; no voters:
I20260810 14:54:03.567950 9529 leader_election.cc:288] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:54:03.567986 9535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:54:03.568122 9535 raft_consensus.cc:686] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [term 1 LEADER]: Becoming Leader. State: Replica: d64668f5d23248ecb467bcfa78ec1576, State: Running, Role: LEADER
I20260810 14:54:03.568219 9535 consensus_queue.cc:237] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64668f5d23248ecb467bcfa78ec1576" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44277 } }
I20260810 14:54:03.568300 9529 sys_catalog.cc:564] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:54:03.568600 9535 sys_catalog.cc:455] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d64668f5d23248ecb467bcfa78ec1576" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64668f5d23248ecb467bcfa78ec1576" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44277 } } }
I20260810 14:54:03.568694 9535 sys_catalog.cc:458] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [sys.catalog]: This master's current role is: LEADER
I20260810 14:54:03.568830 9535 sys_catalog.cc:455] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d64668f5d23248ecb467bcfa78ec1576. Latest consensus state: current_term: 1 leader_uuid: "d64668f5d23248ecb467bcfa78ec1576" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64668f5d23248ecb467bcfa78ec1576" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44277 } } }
I20260810 14:54:03.568903 9535 sys_catalog.cc:458] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576 [sys.catalog]: This master's current role is: LEADER
I20260810 14:54:03.569029 9541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:54:03.569573 9541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:54:03.570590 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:03.558936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47380 (local address 127.30.74.126:44277)
0810 14:54:03.559144 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:03.559153 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:03.559189 (+ 36us) server_negotiation.cc:407] Connection header received
0810 14:54:03.559764 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:03.559774 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:03.559841 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:03.560000 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:03.560687 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:03.562086 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:03.562913 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:03.563580 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:03.563722 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:03.566348 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:03.566352 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:03.566352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:54:03.566364 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:03.568626 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:03.569099 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:03.569102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:03.569102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:03.569214 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:03.569471 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:03.569471 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:03.569478 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:03.569812 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:03.570282 (+ 470us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:03.570339 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 14:54:03.570443 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":86,"threads_started":1}
I20260810 14:54:03.571585 9541 catalog_manager.cc:1347] Generated new cluster ID: 8926b0b30eeb42aeba8f6b52bb5ef9f7
I20260810 14:54:03.571637 9541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:54:03.602735 9541 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:54:03.603355 9541 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:54:03.609167 9541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576: Generated new TSK 0
I20260810 14:54:03.609438 9541 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:54:03.650899 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:44277
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:54:03.753626 9558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:54:03.753849 9558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:54:03.753875 9558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:54:03.754987 9558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:54:03.755023 9558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:54:03.755048 9558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:54:03.756462 9558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:44277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:54:03.756660 9558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:54:03.756844 9558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:54:03.762282 9563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:03.762279 9564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:03.762465 9558 server_base.cc:1034] running on GCE node
W20260810 14:54:03.762285 9566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:03.762830 9558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:54:03.763080 9558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:54:03.764243 9558 hybrid_clock.cc:648] HybridClock initialized: now 1786373643764196 us; error 62 us; skew 500 ppm
Aug 10 14:54:03 dist-test-slave-0b58 krb5kdc[9442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373643, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:54:03.766984 9558 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:54:03.767429 9558 webserver.cc:460] Webserver started at http://127.30.74.65:41111/ using document root <none> and password file <none>
I20260810 14:54:03.767657 9558 fs_manager.cc:359] Metadata directory not provided
I20260810 14:54:03.767709 9558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:54:03.767875 9558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:54:03.769032 9558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "0c72029448ef42eaba7759af3399cf85"
format_stamp: "Formatted at 2026-08-10 14:54:03 on dist-test-slave-0b58"
server_key: "b6cae9b0ba746fd6eac0b5eec660cb25"
server_key_iv: "a1e8ce154f843b400c91f18363378464"
server_key_version: "encryptionkey@0"
I20260810 14:54:03.769423 9558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "0c72029448ef42eaba7759af3399cf85"
format_stamp: "Formatted at 2026-08-10 14:54:03 on dist-test-slave-0b58"
server_key: "b6cae9b0ba746fd6eac0b5eec660cb25"
server_key_iv: "a1e8ce154f843b400c91f18363378464"
server_key_version: "encryptionkey@0"
I20260810 14:54:03.771510 9558 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260810 14:54:03.772508 9573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:03.772722 9558 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:54:03.772840 9558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "0c72029448ef42eaba7759af3399cf85"
format_stamp: "Formatted at 2026-08-10 14:54:03 on dist-test-slave-0b58"
server_key: "b6cae9b0ba746fd6eac0b5eec660cb25"
server_key_iv: "a1e8ce154f843b400c91f18363378464"
server_key_version: "encryptionkey@0"
I20260810 14:54:03.772902 9558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:54:03.798276 9558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:54:03.800349 9558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:54:03.800472 9558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:54:03.800680 9558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:54:03.801033 9558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:54:03.801069 9558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:03.801122 9558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:54:03.801149 9558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:03.807013 9558 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:40333
I20260810 14:54:03.807102 9693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:40333 every 8 connection(s)
I20260810 14:54:03.807448 9558 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:54:03.807960 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 9558
I20260810 14:54:03.808079 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:54:03.808357 31017 external_mini_cluster.cc:1428] Setting key 9ce0c39a905e45fcc0ea9fc4ec4ae10f
Aug 10 14:54:03 dist-test-slave-0b58 krb5kdc[9442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373643, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
I20260810 14:54:03.819303 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:03.808334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:41177 (local address 127.30.74.126:44277)
0810 14:54:03.808402 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:03.808406 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:03.808727 (+ 321us) server_negotiation.cc:407] Connection header received
0810 14:54:03.809256 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:03.809257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:03.809271 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:03.809332 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:03.810324 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:03.811064 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:03.812795 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:03.813138 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:03.813225 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:03.815874 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:03.815876 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:03.815877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:54:03.815888 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:03.817779 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:03.818270 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:03.818272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:03.818273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:03.818344 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:03.818615 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:03.818615 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:03.818616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:03.818733 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:03.818797 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:03.819063 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 14:54:03.819175 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 14:54:03.819463 9696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:03.808342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:44277 (local address 127.30.74.65:41177)
0810 14:54:03.808580 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:03.808637 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:03.809152 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:03.809393 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:03.809396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:03.809998 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:54:03.810214 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:03.810219 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:03.811219 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:03.811220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:03.812676 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:03.812679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:03.813253 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:03.813254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:03.813375 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:03.814023 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:54:03.814033 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:54:03.815757 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:54:03.817878 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:54:03.817885 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:54:03.817887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:54:03.818126 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:54:03.818417 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:54:03.818418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:54:03.818418 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:54:03.818537 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:54:03.818803 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:54:03.818807 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:54:03.818942 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 14:54:03.819032 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 14:54:03.819237 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":84,"threads_started":1}
I20260810 14:54:03.820019 9694 heartbeater.cc:347] Connected to a master server at 127.30.74.126:44277
I20260810 14:54:03.820173 9694 heartbeater.cc:464] Registering TS with master...
I20260810 14:54:03.820456 9694 heartbeater.cc:511] Master 127.30.74.126:44277 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
I20260810 14:54:03.821157 9489 ts_manager.cc:194] Registered new tserver with Master: 0c72029448ef42eaba7759af3399cf85 (127.30.74.65:40333)
I20260810 14:54:03.822103 9489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:41177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:54:03.837247 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:44277
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:54:03.926914 9699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:54:03.927145 9699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:54:03.927172 9699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:54:03.928381 9699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:54:03.928423 9699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:54:03.928450 9699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:54:03.929947 9699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:44277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:54:03.930172 9699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:54:03.930374 9699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:54:03.935655 9704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:03.935655 9707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:03.935724 9699 server_base.cc:1034] running on GCE node
W20260810 14:54:03.935673 9705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:03.936226 9699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:54:03.936547 9699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:54:03.937732 9699 hybrid_clock.cc:648] HybridClock initialized: now 1786373643937717 us; error 29 us; skew 500 ppm
Aug 10 14:54:03 dist-test-slave-0b58 krb5kdc[9442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373643, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:54:03.940316 9699 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:54:03.940841 9699 webserver.cc:460] Webserver started at http://127.30.74.66:46491/ using document root <none> and password file <none>
I20260810 14:54:03.941098 9699 fs_manager.cc:359] Metadata directory not provided
I20260810 14:54:03.941169 9699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:54:03.941320 9699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:54:03.942476 9699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "c10851a6432a4ff7b64f3d7fdfac8260"
format_stamp: "Formatted at 2026-08-10 14:54:03 on dist-test-slave-0b58"
server_key: "d9fc441d207ee7ba5abdfdf1b4b2d9ff"
server_key_iv: "4a288ee896d9905e0d6b36ebd09f8758"
server_key_version: "encryptionkey@0"
I20260810 14:54:03.942879 9699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "c10851a6432a4ff7b64f3d7fdfac8260"
format_stamp: "Formatted at 2026-08-10 14:54:03 on dist-test-slave-0b58"
server_key: "d9fc441d207ee7ba5abdfdf1b4b2d9ff"
server_key_iv: "4a288ee896d9905e0d6b36ebd09f8758"
server_key_version: "encryptionkey@0"
I20260810 14:54:03.944507 9699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:54:03.945367 9715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:03.945554 9699 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:54:03.945667 9699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "c10851a6432a4ff7b64f3d7fdfac8260"
format_stamp: "Formatted at 2026-08-10 14:54:03 on dist-test-slave-0b58"
server_key: "d9fc441d207ee7ba5abdfdf1b4b2d9ff"
server_key_iv: "4a288ee896d9905e0d6b36ebd09f8758"
server_key_version: "encryptionkey@0"
I20260810 14:54:03.945770 9699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:54:03.961158 9699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:54:03.963371 9699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:54:03.963552 9699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:54:03.963763 9699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:54:03.964095 9699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:54:03.964130 9699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:03.964154 9699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:54:03.964167 9699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:03.971019 9699 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:34311
I20260810 14:54:03.971063 9833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:34311 every 8 connection(s)
I20260810 14:54:03.971436 9699 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:54:03.973748 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 9699
I20260810 14:54:03.973886 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:54:03.974076 31017 external_mini_cluster.cc:1428] Setting key f3d66e370a54cd907097d7db9e98f3d5
Aug 10 14:54:03 dist-test-slave-0b58 krb5kdc[9442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373643, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:54:03.982523 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:03.972321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:50819 (local address 127.30.74.126:44277)
0810 14:54:03.972370 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:03.972372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:03.972566 (+ 194us) server_negotiation.cc:407] Connection header received
0810 14:54:03.973101 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:03.973103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:03.973113 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:03.973166 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:03.974230 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:03.974845 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:03.975933 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:03.976136 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:03.976214 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:03.978873 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:03.978875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:03.978875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:54:03.978885 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:03.980897 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:03.981391 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:03.981393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:03.981393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:03.981466 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:03.981751 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:03.981752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:03.981753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:03.981919 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:03.981995 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:03.982299 (+ 304us) server_negotiation.cc:299] Negotiation successful
0810 14:54:03.982393 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 14:54:03.982569 9836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:03.972248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:44277 (local address 127.30.74.66:50819)
0810 14:54:03.972454 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:03.972492 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:03.973011 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:03.973244 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:03.973247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:03.973838 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:54:03.974102 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:03.974108 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:03.974925 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:03.974926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:03.975835 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:03.975838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:03.976223 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:03.976225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:03.976385 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:03.977023 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:54:03.977031 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:54:03.978755 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:54:03.981017 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:54:03.981026 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:54:03.981028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:54:03.981303 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:54:03.981542 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:54:03.981543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:54:03.981543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:54:03.981681 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:54:03.981990 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:54:03.981995 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:54:03.982158 (+ 163us) client_negotiation.cc:769] Sending connection context
0810 14:54:03.982248 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 14:54:03.982391 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260810 14:54:03.983209 9834 heartbeater.cc:347] Connected to a master server at 127.30.74.126:44277
I20260810 14:54:03.983305 9834 heartbeater.cc:464] Registering TS with master...
I20260810 14:54:03.983527 9834 heartbeater.cc:511] Master 127.30.74.126:44277 requested a full tablet report, sending...
I20260810 14:54:03.983989 9489 ts_manager.cc:194] Registered new tserver with Master: c10851a6432a4ff7b64f3d7fdfac8260 (127.30.74.66:34311)
I20260810 14:54:03.984434 9489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:50819
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:54:04.004212 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:44277
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:54:04.089818 9839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:54:04.090070 9839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:54:04.090126 9839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:54:04.091321 9839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:54:04.091398 9839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:54:04.091445 9839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:54:04.092985 9839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:44277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:54:04.093268 9839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:54:04.093514 9839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:54:04.098922 9845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:04.098889 9847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:04.098934 9839 server_base.cc:1034] running on GCE node
W20260810 14:54:04.098896 9844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:04.099339 9839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:54:04.099627 9839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:54:04.100782 9839 hybrid_clock.cc:648] HybridClock initialized: now 1786373644100768 us; error 27 us; skew 500 ppm
Aug 10 14:54:04 dist-test-slave-0b58 krb5kdc[9442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373644, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:54:04.103359 9839 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:54:04.103826 9839 webserver.cc:460] Webserver started at http://127.30.74.67:42867/ using document root <none> and password file <none>
I20260810 14:54:04.104038 9839 fs_manager.cc:359] Metadata directory not provided
I20260810 14:54:04.104086 9839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:54:04.104182 9839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:54:04.105340 9839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "fb42bf92b07049bf9ae2559a845199c7"
format_stamp: "Formatted at 2026-08-10 14:54:04 on dist-test-slave-0b58"
server_key: "8f04f62711f6d92a4f1db03a54ff4f09"
server_key_iv: "5f7b3ae93e0d42cbec71c1b03256f2ef"
server_key_version: "encryptionkey@0"
I20260810 14:54:04.105758 9839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "fb42bf92b07049bf9ae2559a845199c7"
format_stamp: "Formatted at 2026-08-10 14:54:04 on dist-test-slave-0b58"
server_key: "8f04f62711f6d92a4f1db03a54ff4f09"
server_key_iv: "5f7b3ae93e0d42cbec71c1b03256f2ef"
server_key_version: "encryptionkey@0"
I20260810 14:54:04.107160 9839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:54:04.107983 9855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:04.108249 9839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:54:04.108402 9839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "fb42bf92b07049bf9ae2559a845199c7"
format_stamp: "Formatted at 2026-08-10 14:54:04 on dist-test-slave-0b58"
server_key: "8f04f62711f6d92a4f1db03a54ff4f09"
server_key_iv: "5f7b3ae93e0d42cbec71c1b03256f2ef"
server_key_version: "encryptionkey@0"
I20260810 14:54:04.108489 9839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:54:04.139418 9839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:54:04.141880 9839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:54:04.142074 9839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:54:04.142385 9839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:54:04.142726 9839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:54:04.142779 9839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:04.142822 9839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:54:04.142863 9839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:04.149271 9839 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:39263
I20260810 14:54:04.149318 9974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:39263 every 8 connection(s)
I20260810 14:54:04.149675 9839 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:54:04.150384 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 9839
I20260810 14:54:04.150486 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:54:04.150715 31017 external_mini_cluster.cc:1428] Setting key a52edc0d3bdcf30065379a107ed56523
Aug 10 14:54:04 dist-test-slave-0b58 krb5kdc[9442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373644, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:54:04.159781 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:04.150599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:40193 (local address 127.30.74.126:44277)
0810 14:54:04.150650 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:04.150652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:04.150962 (+ 310us) server_negotiation.cc:407] Connection header received
0810 14:54:04.151665 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:04.151666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:04.151680 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:04.151744 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:04.152821 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.153340 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.154369 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.154583 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.154662 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:04.156951 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:04.156953 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:04.156953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:54:04.156962 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:04.158333 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:04.158760 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:04.158761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:04.158762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:04.158834 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:04.159091 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:04.159093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:04.159093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:04.159222 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:04.159297 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:04.159589 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 14:54:04.159677 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:54:04.159994 9977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:04.150557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:44277 (local address 127.30.74.67:40193)
0810 14:54:04.150742 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:04.150810 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:04.151548 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:04.151816 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:04.151820 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:04.152496 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:54:04.152719 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:04.152724 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.153444 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.153446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:04.154273 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:04.154275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.154672 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.154674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:04.154803 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:04.155385 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:54:04.155394 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:54:04.156842 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:54:04.158427 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:54:04.158435 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:54:04.158438 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:54:04.158677 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:54:04.158906 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:54:04.158907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:54:04.158907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:54:04.159022 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:54:04.159301 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:54:04.159308 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:54:04.159454 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 14:54:04.159566 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 14:54:04.159739 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":105,"threads_started":1}
I20260810 14:54:04.160755 9975 heartbeater.cc:347] Connected to a master server at 127.30.74.126:44277
I20260810 14:54:04.160861 9975 heartbeater.cc:464] Registering TS with master...
I20260810 14:54:04.161077 9975 heartbeater.cc:511] Master 127.30.74.126:44277 requested a full tablet report, sending...
I20260810 14:54:04.161540 9489 ts_manager.cc:194] Registered new tserver with Master: fb42bf92b07049bf9ae2559a845199c7 (127.30.74.67:39263)
I20260810 14:54:04.161934 9489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:40193
I20260810 14:54:04.162721 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:54:04.170537 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:04.164088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47396 (local address 127.30.74.126:44277)
0810 14:54:04.164153 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:04.164155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:04.164288 (+ 133us) server_negotiation.cc:407] Connection header received
0810 14:54:04.164380 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:04.164381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:04.164391 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:04.164448 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:04.165006 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.165487 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.166228 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.166434 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.166515 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:04.167723 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:04.167724 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:04.167725 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:54:04.167732 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:04.169202 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:04.169604 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:04.169605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:04.169606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:04.169671 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:04.169912 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:04.169913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:04.169914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:04.170077 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:04.170189 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:04.170345 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 14:54:04.170430 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 14:54:04.172119 9489 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:54:04.174235 9489 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:54:04.181475 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:04.177316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60322 (local address 127.30.74.67:39263)
0810 14:54:04.177564 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:04.177569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:04.177585 (+ 16us) server_negotiation.cc:407] Connection header received
0810 14:54:04.177675 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:04.177677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:04.177690 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:04.177793 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:54:04.177993 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.178864 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.180216 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.180995 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.181088 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:04.181151 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:04.181330 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 14:54:04.181380 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":99,"threads_started":1}
I20260810 14:54:04.181468 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:04.176977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:40333 (local address 127.0.0.1:56538)
0810 14:54:04.177146 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:04.177194 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:04.177299 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:04.177874 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:04.177875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:04.177878 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:54:04.177932 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:04.177933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.179275 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.179276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:04.180264 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:04.180267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.181186 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.181187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:04.181256 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:04.181260 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:54:04.181311 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 14:54:04.181394 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":19,"thread_start_us":55,"threads_started":1}
I20260810 14:54:04.181507 9991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:04.177073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56538 (local address 127.30.74.65:40333)
0810 14:54:04.177616 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:04.177621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:04.177633 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:54:04.177675 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:04.177677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:04.177692 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:04.177791 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:54:04.178022 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.179153 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.180362 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.181014 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.181089 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:04.181150 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:04.181373 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 14:54:04.181419 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":129,"threads_started":1}
I20260810 14:54:04.181982 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:04.177199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:34311 (local address 127.0.0.1:51536)
0810 14:54:04.177903 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:04.177913 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:04.177962 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:04.178250 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:04.178252 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:04.178254 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:54:04.178307 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:04.178309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.179618 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.179620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:04.180714 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:04.180717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.181746 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.181747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:04.181818 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:04.181822 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:54:04.181874 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 14:54:04.181913 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":105,"threads_started":1}
I20260810 14:54:04.181991 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:04.177382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51536 (local address 127.30.74.66:34311)
0810 14:54:04.177585 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:04.177589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:04.178024 (+ 435us) server_negotiation.cc:407] Connection header received
0810 14:54:04.178075 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:04.178078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:04.178094 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:04.178162 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:54:04.178402 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.179481 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.180819 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.181642 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.181710 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:04.181749 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:04.181897 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 14:54:04.181932 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":98,"threads_started":1}
I20260810 14:54:04.182466 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:04.176993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:39263 (local address 127.0.0.1:60322)
0810 14:54:04.177518 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:04.177530 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:04.177591 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:04.177793 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:04.177796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:04.177798 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:54:04.177895 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:04.177901 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.178977 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.178978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:04.180108 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:04.180111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.181090 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.181092 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:04.181186 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:04.181201 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 14:54:04.181308 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 14:54:04.182402 (+ 1094us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":91,"threads_started":1}
I20260810 14:54:04.182834 9624 tablet_service.cc:1467] Processing CreateTablet for tablet 39394cb10b8742c38229a6af8c1d3d0d (DEFAULT_TABLE table=test-table [id=a40e08dc1de548a9961a9e3f0d68cc80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:54:04.182945 9766 tablet_service.cc:1467] Processing CreateTablet for tablet 39394cb10b8742c38229a6af8c1d3d0d (DEFAULT_TABLE table=test-table [id=a40e08dc1de548a9961a9e3f0d68cc80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:54:04.183269 9624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39394cb10b8742c38229a6af8c1d3d0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:54:04.183296 9766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39394cb10b8742c38229a6af8c1d3d0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:54:04.183338 9905 tablet_service.cc:1467] Processing CreateTablet for tablet 39394cb10b8742c38229a6af8c1d3d0d (DEFAULT_TABLE table=test-table [id=a40e08dc1de548a9961a9e3f0d68cc80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:54:04.183660 9905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39394cb10b8742c38229a6af8c1d3d0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:54:04.186161 9995 tablet_bootstrap.cc:492] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260: Bootstrap starting.
I20260810 14:54:04.186858 9996 tablet_bootstrap.cc:492] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85: Bootstrap starting.
I20260810 14:54:04.187142 9995 tablet_bootstrap.cc:654] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260: Neither blocks nor log segments found. Creating new log.
I20260810 14:54:04.187296 9997 tablet_bootstrap.cc:492] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7: Bootstrap starting.
I20260810 14:54:04.187588 9996 tablet_bootstrap.cc:654] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85: Neither blocks nor log segments found. Creating new log.
I20260810 14:54:04.187920 9997 tablet_bootstrap.cc:654] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7: Neither blocks nor log segments found. Creating new log.
I20260810 14:54:04.187883 9995 log.cc:824] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260: Log is configured to *not* fsync() on all Append() calls
I20260810 14:54:04.188169 9996 log.cc:824] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85: Log is configured to *not* fsync() on all Append() calls
I20260810 14:54:04.188526 9997 log.cc:824] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7: Log is configured to *not* fsync() on all Append() calls
I20260810 14:54:04.188882 9995 tablet_bootstrap.cc:492] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260: No bootstrap required, opened a new log
I20260810 14:54:04.188973 9995 ts_tablet_manager.cc:1397] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 14:54:04.189023 9996 tablet_bootstrap.cc:492] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85: No bootstrap required, opened a new log
I20260810 14:54:04.189097 9996 ts_tablet_manager.cc:1397] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:54:04.189292 9997 tablet_bootstrap.cc:492] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7: No bootstrap required, opened a new log
I20260810 14:54:04.189363 9997 ts_tablet_manager.cc:1397] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:54:04.190752 9996 raft_consensus.cc:348] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c72029448ef42eaba7759af3399cf85" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40333 } } peers { permanent_uuid: "fb42bf92b07049bf9ae2559a845199c7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39263 } } peers { permanent_uuid: "c10851a6432a4ff7b64f3d7fdfac8260" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34311 } }
I20260810 14:54:04.190971 9996 raft_consensus.cc:374] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:54:04.191021 9996 raft_consensus.cc:729] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c72029448ef42eaba7759af3399cf85, State: Initialized, Role: FOLLOWER
I20260810 14:54:04.191139 9996 consensus_queue.cc:260] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c72029448ef42eaba7759af3399cf85" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40333 } } peers { permanent_uuid: "fb42bf92b07049bf9ae2559a845199c7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39263 } } peers { permanent_uuid: "c10851a6432a4ff7b64f3d7fdfac8260" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34311 } }
I20260810 14:54:04.191077 9995 raft_consensus.cc:348] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c72029448ef42eaba7759af3399cf85" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40333 } } peers { permanent_uuid: "fb42bf92b07049bf9ae2559a845199c7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39263 } } peers { permanent_uuid: "c10851a6432a4ff7b64f3d7fdfac8260" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34311 } }
I20260810 14:54:04.191238 9995 raft_consensus.cc:374] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:54:04.191284 9995 raft_consensus.cc:729] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c10851a6432a4ff7b64f3d7fdfac8260, State: Initialized, Role: FOLLOWER
I20260810 14:54:04.191411 9995 consensus_queue.cc:260] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c72029448ef42eaba7759af3399cf85" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40333 } } peers { permanent_uuid: "fb42bf92b07049bf9ae2559a845199c7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39263 } } peers { permanent_uuid: "c10851a6432a4ff7b64f3d7fdfac8260" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34311 } }
I20260810 14:54:04.191453 9996 ts_tablet_manager.cc:1428] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 14:54:04.191390 9997 raft_consensus.cc:348] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c72029448ef42eaba7759af3399cf85" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40333 } } peers { permanent_uuid: "fb42bf92b07049bf9ae2559a845199c7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39263 } } peers { permanent_uuid: "c10851a6432a4ff7b64f3d7fdfac8260" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34311 } }
I20260810 14:54:04.191587 9694 heartbeater.cc:503] Master 127.30.74.126:44277 was elected leader, sending a full tablet report...
I20260810 14:54:04.191705 9995 ts_tablet_manager.cc:1428] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:54:04.191731 9834 heartbeater.cc:503] Master 127.30.74.126:44277 was elected leader, sending a full tablet report...
I20260810 14:54:04.191851 9997 raft_consensus.cc:374] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:54:04.191955 9997 raft_consensus.cc:729] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb42bf92b07049bf9ae2559a845199c7, State: Initialized, Role: FOLLOWER
I20260810 14:54:04.192068 9997 consensus_queue.cc:260] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c72029448ef42eaba7759af3399cf85" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40333 } } peers { permanent_uuid: "fb42bf92b07049bf9ae2559a845199c7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39263 } } peers { permanent_uuid: "c10851a6432a4ff7b64f3d7fdfac8260" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34311 } }
I20260810 14:54:04.192394 9997 ts_tablet_manager.cc:1428] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 14:54:04.192414 9975 heartbeater.cc:503] Master 127.30.74.126:44277 was elected leader, sending a full tablet report...
W20260810 14:54:04.222046 9835 tablet.cc:2369] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:54:04.308341 9695 tablet.cc:2369] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:54:04.357998 10001 raft_consensus.cc:482] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:54:04.358135 10001 raft_consensus.cc:504] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c72029448ef42eaba7759af3399cf85" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40333 } } peers { permanent_uuid: "fb42bf92b07049bf9ae2559a845199c7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39263 } } peers { permanent_uuid: "c10851a6432a4ff7b64f3d7fdfac8260" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34311 } }
I20260810 14:54:04.358449 10001 leader_election.cc:288] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb42bf92b07049bf9ae2559a845199c7 (127.30.74.67:39263), c10851a6432a4ff7b64f3d7fdfac8260 (127.30.74.66:34311)
I20260810 14:54:04.361898 10004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:04.358671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:39263 (local address 127.30.74.65:42479)
0810 14:54:04.358832 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:04.358845 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:04.359055 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:04.359243 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:04.359244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:04.359247 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:54:04.359329 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:04.359332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.360003 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.360004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:04.361010 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:04.361013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.361598 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.361599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:04.361680 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:04.361684 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:54:04.361753 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 14:54:04.361805 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260810 14:54:04.361884 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:04.358857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:42479 (local address 127.30.74.67:39263)
0810 14:54:04.358930 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:04.358933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:04.358947 (+ 14us) server_negotiation.cc:407] Connection header received
0810 14:54:04.359162 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:04.359163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:04.359173 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:04.359239 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:54:04.359422 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.359913 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.361105 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.361509 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.361600 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:04.361610 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:04.361760 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 14:54:04.361806 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 14:54:04.362017 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:04.359123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:38493 (local address 127.30.74.66:34311)
0810 14:54:04.359174 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:04.359176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:04.359184 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:54:04.359221 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:04.359222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:04.359235 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:04.359296 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:54:04.359478 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.360074 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.361211 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.361603 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.361697 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:04.361706 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:04.361887 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 14:54:04.361930 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 14:54:04.362249 10005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:04.358928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:34311 (local address 127.30.74.65:38493)
0810 14:54:04.359019 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:04.359028 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:04.359121 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:04.359304 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:04.359305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:04.359306 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:54:04.359398 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:04.359400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.360178 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.360179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:04.361099 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:04.361102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.361696 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.361697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:04.361783 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:04.361796 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 14:54:04.361883 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 14:54:04.361928 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":54,"threads_started":1}
I20260810 14:54:04.362502 9926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39394cb10b8742c38229a6af8c1d3d0d" candidate_uuid: "0c72029448ef42eaba7759af3399cf85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb42bf92b07049bf9ae2559a845199c7" is_pre_election: true
I20260810 14:54:04.362598 9786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39394cb10b8742c38229a6af8c1d3d0d" candidate_uuid: "0c72029448ef42eaba7759af3399cf85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c10851a6432a4ff7b64f3d7fdfac8260" is_pre_election: true
I20260810 14:54:04.362742 9926 raft_consensus.cc:2454] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c72029448ef42eaba7759af3399cf85 in term 0.
I20260810 14:54:04.362811 9786 raft_consensus.cc:2454] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c72029448ef42eaba7759af3399cf85 in term 0.
I20260810 14:54:04.363031 9576 leader_election.cc:302] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c72029448ef42eaba7759af3399cf85, fb42bf92b07049bf9ae2559a845199c7; no voters:
I20260810 14:54:04.363199 10001 raft_consensus.cc:2781] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:54:04.363261 10001 raft_consensus.cc:482] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:54:04.363288 10001 raft_consensus.cc:3037] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:54:04.364066 10001 raft_consensus.cc:504] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c72029448ef42eaba7759af3399cf85" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40333 } } peers { permanent_uuid: "fb42bf92b07049bf9ae2559a845199c7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39263 } } peers { permanent_uuid: "c10851a6432a4ff7b64f3d7fdfac8260" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34311 } }
I20260810 14:54:04.364223 10001 leader_election.cc:288] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [CANDIDATE]: Term 1 election: Requested vote from peers fb42bf92b07049bf9ae2559a845199c7 (127.30.74.67:39263), c10851a6432a4ff7b64f3d7fdfac8260 (127.30.74.66:34311)
I20260810 14:54:04.364441 9926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39394cb10b8742c38229a6af8c1d3d0d" candidate_uuid: "0c72029448ef42eaba7759af3399cf85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb42bf92b07049bf9ae2559a845199c7"
I20260810 14:54:04.364444 9786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39394cb10b8742c38229a6af8c1d3d0d" candidate_uuid: "0c72029448ef42eaba7759af3399cf85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c10851a6432a4ff7b64f3d7fdfac8260"
I20260810 14:54:04.364524 9926 raft_consensus.cc:3037] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:54:04.364542 9786 raft_consensus.cc:3037] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:54:04.365267 9926 raft_consensus.cc:2454] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c72029448ef42eaba7759af3399cf85 in term 1.
I20260810 14:54:04.365268 9786 raft_consensus.cc:2454] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c72029448ef42eaba7759af3399cf85 in term 1.
I20260810 14:54:04.365468 9576 leader_election.cc:302] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c72029448ef42eaba7759af3399cf85, c10851a6432a4ff7b64f3d7fdfac8260; no voters:
I20260810 14:54:04.365574 10001 raft_consensus.cc:2781] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:54:04.365751 10001 raft_consensus.cc:686] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [term 1 LEADER]: Becoming Leader. State: Replica: 0c72029448ef42eaba7759af3399cf85, State: Running, Role: LEADER
I20260810 14:54:04.365986 10001 consensus_queue.cc:237] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c72029448ef42eaba7759af3399cf85" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40333 } } peers { permanent_uuid: "fb42bf92b07049bf9ae2559a845199c7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39263 } } peers { permanent_uuid: "c10851a6432a4ff7b64f3d7fdfac8260" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34311 } }
I20260810 14:54:04.366911 9489 catalog_manager.cc:5495] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c72029448ef42eaba7759af3399cf85 (127.30.74.65). New cstate: current_term: 1 leader_uuid: "0c72029448ef42eaba7759af3399cf85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c72029448ef42eaba7759af3399cf85" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb42bf92b07049bf9ae2559a845199c7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c10851a6432a4ff7b64f3d7fdfac8260" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34311 } health_report { overall_health: UNKNOWN } } }
W20260810 14:54:04.400388 9976 tablet.cc:2369] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:54:04.410055 9991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:04.407550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56550 (local address 127.30.74.65:40333)
0810 14:54:04.407623 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:04.407625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:04.407635 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:54:04.407688 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:04.407689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:04.407715 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:04.407809 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:54:04.407961 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.408461 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.409250 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:04.409451 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:04.409528 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:04.409699 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:54:04.409812 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:54:04.409909 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:04.409933 (+ 24us) server_negotiation.cc:299] Negotiation successful
0810 14:54:04.409955 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 14:54:04.412016 9786 raft_consensus.cc:1261] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260 [term 1 FOLLOWER]: Refusing update from remote peer 0c72029448ef42eaba7759af3399cf85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:54:04.412017 9926 raft_consensus.cc:1261] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7 [term 1 FOLLOWER]: Refusing update from remote peer 0c72029448ef42eaba7759af3399cf85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:54:04.412428 10001 consensus_queue.cc:1035] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c10851a6432a4ff7b64f3d7fdfac8260" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 34311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:54:04.412577 10006 consensus_queue.cc:1035] T 39394cb10b8742c38229a6af8c1d3d0d P 0c72029448ef42eaba7759af3399cf85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb42bf92b07049bf9ae2559a845199c7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:54:04.415433 10011 mvcc.cc:205] Tried to move back new op lower bound from 7316986447508664320 to 7316986447323934720. Current Snapshot: MvccSnapshot[applied={T|T < 7316986447508664320}]
I20260810 14:54:04.415472 10010 mvcc.cc:205] Tried to move back new op lower bound from 7316986447508664320 to 7316986447323934720. Current Snapshot: MvccSnapshot[applied={T|T < 7316986447508664320}]
I20260810 14:54:04.417665 9489 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:54:04.417781 9489 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:54:04.418613 9489 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d64668f5d23248ecb467bcfa78ec1576: Sending DeleteTablet for 3 replicas of tablet 39394cb10b8742c38229a6af8c1d3d0d
I20260810 14:54:04.418958 9624 tablet_service.cc:1514] Processing DeleteTablet for tablet 39394cb10b8742c38229a6af8c1d3d0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:54:04 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:56538
I20260810 14:54:04.419005 9766 tablet_service.cc:1514] Processing DeleteTablet for tablet 39394cb10b8742c38229a6af8c1d3d0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:54:04 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:51536
I20260810 14:54:04.419273 10021 tablet_replica.cc:330] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260: stopping tablet replica
I20260810 14:54:04.419335 9905 tablet_service.cc:1514] Processing DeleteTablet for tablet 39394cb10b8742c38229a6af8c1d3d0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:54:04 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:60322
I20260810 14:54:04.419492 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 9558
I20260810 14:54:04.419494 10021 raft_consensus.cc:2229] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:54:04.420208 10021 raft_consensus.cc:2258] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:54:04.421066 10021 ts_tablet_manager.cc:1905] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:54:04.421833 10022 tablet_replica.cc:330] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7: stopping tablet replica
I20260810 14:54:04.421955 10022 raft_consensus.cc:2229] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:54:04.422070 10022 raft_consensus.cc:2258] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:54:04.422547 10022 ts_tablet_manager.cc:1905] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:54:04.423255 10021 ts_tablet_manager.cc:1918] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:54:04.423396 10021 log.cc:1196] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/39394cb10b8742c38229a6af8c1d3d0d
I20260810 14:54:04.423791 10021 ts_tablet_manager.cc:1939] T 39394cb10b8742c38229a6af8c1d3d0d P c10851a6432a4ff7b64f3d7fdfac8260: Deleting consensus metadata
I20260810 14:54:04.424218 10022 ts_tablet_manager.cc:1918] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:54:04.424314 10022 log.cc:1196] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/39394cb10b8742c38229a6af8c1d3d0d
I20260810 14:54:04.424340 9475 catalog_manager.cc:4841] TS c10851a6432a4ff7b64f3d7fdfac8260 (127.30.74.66:34311): tablet 39394cb10b8742c38229a6af8c1d3d0d (table test-table [id=a40e08dc1de548a9961a9e3f0d68cc80]) successfully deleted
I20260810 14:54:04.424597 10022 ts_tablet_manager.cc:1939] T 39394cb10b8742c38229a6af8c1d3d0d P fb42bf92b07049bf9ae2559a845199c7: Deleting consensus metadata
I20260810 14:54:04.425020 9475 catalog_manager.cc:4841] TS fb42bf92b07049bf9ae2559a845199c7 (127.30.74.67:39263): tablet 39394cb10b8742c38229a6af8c1d3d0d (table test-table [id=a40e08dc1de548a9961a9e3f0d68cc80]) successfully deleted
W20260810 14:54:04.428047 9476 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:40333 (error 108)
I20260810 14:54:04.428222 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 9699
I20260810 14:54:04.430565 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:04.430319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:40333 (local address 127.0.0.1:56552)
0810 14:54:04.430401 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:04.430478 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:40333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 14:54:04.430790 9476 catalog_manager.cc:4584] TS 0c72029448ef42eaba7759af3399cf85 (127.30.74.65:40333): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 39394cb10b8742c38229a6af8c1d3d0d: Network error: Client connection negotiation failed: client connection to 127.30.74.65:40333: connect: Connection refused (error 111)
I20260810 14:54:04.441650 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 9839
I20260810 14:54:04.448457 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 9455
2026-08-10T14:54:04Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2237 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:54:04 dist-test-slave-0b58 krb5kdc[10027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:54:04 dist-test-slave-0b58 krb5kdc[10027](info): set up 2 sockets
Aug 10 14:54:04 dist-test-slave-0b58 krb5kdc[10027](info): commencing operation
krb5kdc: starting...
W20260810 14:54:05.547466 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:54:05 dist-test-slave-0b58 krb5kdc[10027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:54:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:54:05Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.74.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.74.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.74.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/rangeradmin_127.30.74.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.74.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/rangeradmin_127.30.74.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.74.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.74.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.74.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/rangerlookup_127.30.74.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.74.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/rangerlookup_127.30.74.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.74.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.74.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.74.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/HTTP_127.30.74.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.74.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/HTTP_127.30.74.84@KRBTEST.COM.keytab.
I20260810 14:54:05.727098 31017 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task2SWuiv/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/postgres -l logfile start
2026-08-10 14:54:08.240 UTC [10057] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 14:54:08.240 UTC [10057] LOG: listening on IPv4 address "127.30.74.84", port 43111
2026-08-10 14:54:08.244 UTC [10057] LOG: listening on Unix socket "/tmp/.s.PGSQL.43111"
2026-08-10 14:54:08.267 UTC [10060] LOG: database system was shut down at 2026-08-10 14:54:06 UTC
2026-08-10 14:54:08.271 UTC [10057] LOG: database system is ready to accept connections
I20260810 14:54:09.256667 31017 mini_postgres.cc:96] Postgres bound to 43111
2026-08-10 14:54:09.263 UTC [10069] FATAL: database "slave" does not exist
127.30.74.84:43111 - accepting connections
I20260810 14:54:09.264289 31017 mini_ranger.cc:161] Starting Ranger...
I20260810 14:54:09.279639 31017 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 14:54:09.732756 31017 mini_ranger.cc:87] Created ranger Postgres database
I20260810 14:54:09.732870 31017 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-admin
2026-08-10 14:54:10,024 [I] DB FLAVOR :POSTGRES
2026-08-10 14:54:10,025 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 14:54:10,025 [I] Checking connection..
2026-08-10 14:54:10,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 14:54:10,289 [I] Checking connection passed.
2026-08-10 14:54:10,290 [I] --------- Verifying version history table ---------
2026-08-10 14:54:10,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:54:10,544 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 14:54:10,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:54:10,780 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 14:54:10,780 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 14:54:10,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 14:54:11.035 UTC [10139] WARNING: there is no transaction in progress
2026-08-10 14:54:11,043 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 14:54:11,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:54:11,292 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 14:54:11,292 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 14:54:11,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 14:54:11,543 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 14:54:11,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-0b58.c.gcp-upstream.internal','N') ;"
2026-08-10 14:54:12,005 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 14:54:12,239 [I] Table x_portal_user does not exist in database ranger
2026-08-10 14:54:12,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 14:54:12,471 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 14:54:12,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 14:54:12,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 14:54:12,977 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 14:54:12,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 14:54:14.037 UTC [10250] WARNING: there is no transaction in progress
2026-08-10 14:54:14.055 UTC [10250] WARNING: there is no transaction in progress
2026-08-10 14:54:14.074 UTC [10250] WARNING: there is no transaction in progress
2026-08-10 14:54:14.092 UTC [10250] WARNING: there is no transaction in progress
2026-08-10 14:54:14.113 UTC [10250] WARNING: there is no transaction in progress
2026-08-10 14:54:14.133 UTC [10250] WARNING: there is no transaction in progress
2026-08-10 14:54:14.198 UTC [10250] WARNING: there is no transaction in progress
2026-08-10 14:54:14.206 UTC [10250] WARNING: there is no transaction in progress
2026-08-10 14:54:14.224 UTC [10250] WARNING: there is no transaction in progress
2026-08-10 14:54:14.236 UTC [10250] WARNING: there is no transaction in progress
2026-08-10 14:54:14.248 UTC [10250] WARNING: there is no transaction in progress
2026-08-10 14:54:14.264 UTC [10250] WARNING: there is no transaction in progress
2026-08-10 14:54:14.275 UTC [10250] WARNING: there is no transaction in progress
2026-08-10 14:54:14.283 UTC [10250] WARNING: there is no transaction in progress
2026-08-10 14:54:14.292 UTC [10250] WARNING: there is no transaction in progress
2026-08-10 14:54:14.301 UTC [10250] WARNING: there is no transaction in progress
2026-08-10 14:54:15,075 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 14:54:15,076 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 14:54:15,344 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 14:54:15,344 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 14:54:15,598 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 14:54:15,599 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 14:54:15,848 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 14:54:15,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 14:54:16,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 14:54:16,348 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-0b58.c.gcp-upstream.internal';"
2026-08-10 14:54:16,611 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 14:54:16,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 14:54:16,853 [I] DB_PATCHES have already been applied
I20260810 14:54:16.859148 31017 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-admin:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task2SWuiv/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 14:54:16.859222 31017 mini_ranger.cc:193] Using host: 127.30.74.84
I20260810 14:54:16.863121 31017 mini_ranger.cc:227] Ranger admin URL: http://127.30.74.84:36937
Aug 10, 2026 2:54:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 2:54:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 2:54:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 2:54:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 2:54:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 2:54:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 2:54:17 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 14:54:18.033145 31017 mini_ranger.cc:160] Time spent starting Ranger: real 8.769s user 0.002s sys 0.009s
Aug 10, 2026 2:54:40 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 2:54:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 10, 2026 2:54:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 2:54:40 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 2:54:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 2:54:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 2:54:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 2:54:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 2:54:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 2:54:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 2:54:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 2:54:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 2:54:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 2:54:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 2:54:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 2:54:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 2:54:40 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 2:54:41 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 14:54:41.709836 31017 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.74.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.74.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.74.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/rangerkms_127.30.74.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.74.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/rangerkms_127.30.74.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.74.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.74.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.74.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/HTTP_127.30.74.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.74.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/HTTP_127.30.74.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 14:54:41 dist-test-slave-0b58 krb5kdc[10027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373681, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 14:54:41.799500 31017 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 14:54:41.810115 31017 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 14:54:42.634979 31017 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 14:54:42.635079 31017 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-kms
I20260810 14:54:42.635100 31017 mini_ranger_kms.cc:227] Using postgres at 127.30.74.84:43111
2026-08-10 14:54:42,846 [I] DB FLAVOR :POSTGRES
2026-08-10 14:54:42,846 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 14:54:42,846 [I] Checking connection
2026-08-10 14:54:42,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 14:54:43,086 [I] connection success
2026-08-10 14:54:43,086 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 14:54:43,087 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 14:54:43,087 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 14:54:43,327 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 14:54:43,327 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 14:54:43,328 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 14:54:43,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:43111/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 14:54:43,603 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 14:54:44.159997 31017 mini_ranger_kms.cc:310] Created kms service
I20260810 14:54:44.375123 31017 mini_ranger_kms.cc:326] Created kudu user
I20260810 14:54:44.419214 31017 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 14:54:44 dist-test-slave-0b58 krb5kdc[10027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373681, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
I20260810 14:54:44.836797 31017 mini_ranger_kms.cc:384] Added ranger policy
I20260810 14:54:44.836939 31017 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-kms:/tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 14:54:44.836966 31017 mini_ranger_kms.cc:240] Using host: 127.30.74.84
I20260810 14:54:44.841274 31017 mini_ranger_kms.cc:276] Ranger KMS PID: 10443
I20260810 14:54:44.841436 31017 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.74.84:32953
Aug 10, 2026 2:54:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 2:54:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 2:54:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 2:54:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 2:54:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 2:54:46 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260810 14:54:47.287774 31017 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.488s user 0.004s sys 0.190s
I20260810 14:54:47.287911 31017 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 14:54:47.288017 31017 mini_ranger_kms.cc:401] 127.30.74.84:32953/kms/v1/keys
Aug 10 14:54:47 dist-test-slave-0b58 krb5kdc[10027](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786373687, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.74.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 14:54:47 dist-test-slave-0b58 krb5kdc[10027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373687, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.74.84@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
Aug 10 14:54:48 dist-test-slave-0b58 krb5kdc[10027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373687, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.74.84@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
Aug 10 14:54:50 dist-test-slave-0b58 krb5kdc[10027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:54:50.603228 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:32775
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:32775
--ranger_config_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.74.84:32953/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:54:50.704355 10497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:54:50.704578 10497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:54:50.704602 10497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:54:50.705971 10497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:54:50.706044 10497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:54:50.706059 10497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:54:50.706072 10497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:54:50.706085 10497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:54:50.708221 10497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.74.84:32953/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:32775
--ranger_config_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:32775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:54:50.708626 10497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:54:50.708927 10497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 14:54:50.714540 10497 server_base.cc:1034] running on GCE node
W20260810 14:54:50.714509 10505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:50.714507 10503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:50.714500 10502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:50.715117 10497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:54:50.715401 10497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:54:50.716571 10497 hybrid_clock.cc:648] HybridClock initialized: now 1786373690716550 us; error 33 us; skew 500 ppm
Aug 10 14:54:50 dist-test-slave-0b58 krb5kdc[10027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373690, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:54:50.723436 10497 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:54:50.724041 10497 webserver.cc:460] Webserver started at http://127.30.74.126:33167/ using document root <none> and password file <none>
I20260810 14:54:50.724362 10497 fs_manager.cc:359] Metadata directory not provided
I20260810 14:54:50.724428 10497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:54:50.724591 10497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:54:50 dist-test-slave-0b58 krb5kdc[10027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373690, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
I20260810 14:54:50.788180 10497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "6005b3b0a8204bffa341cb0c9d064536"
format_stamp: "Formatted at 2026-08-10 14:54:50 on dist-test-slave-0b58"
server_key: "bdd00dee5d29137e65678e08650c04ec"
server_key_iv: "4417a4337ebca70c604efcd3d218a7d7"
server_key_version: "kuduclusterkey@0"
I20260810 14:54:50.788821 10497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "6005b3b0a8204bffa341cb0c9d064536"
format_stamp: "Formatted at 2026-08-10 14:54:50 on dist-test-slave-0b58"
server_key: "bdd00dee5d29137e65678e08650c04ec"
server_key_iv: "4417a4337ebca70c604efcd3d218a7d7"
server_key_version: "kuduclusterkey@0"
I20260810 14:54:50.790892 10497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:54:50.807087 10512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:50.807291 10497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 14:54:50.807410 10497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "6005b3b0a8204bffa341cb0c9d064536"
format_stamp: "Formatted at 2026-08-10 14:54:50 on dist-test-slave-0b58"
server_key: "bdd00dee5d29137e65678e08650c04ec"
server_key_iv: "4417a4337ebca70c604efcd3d218a7d7"
server_key_version: "kuduclusterkey@0"
I20260810 14:54:50.807529 10497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:54:50.828624 10497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:54:50.830626 10497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:54:50.830806 10497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:54:50.835458 10497 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:32775
I20260810 14:54:50.835502 10567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:32775 every 8 connection(s)
I20260810 14:54:50.835896 10497 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:54:50.836632 10568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:54:50.838917 10568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536: Bootstrap starting.
I20260810 14:54:50.839538 10568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536: Neither blocks nor log segments found. Creating new log.
I20260810 14:54:50.839797 10568 log.cc:824] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536: Log is configured to *not* fsync() on all Append() calls
I20260810 14:54:50.840580 10568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536: No bootstrap required, opened a new log
I20260810 14:54:50.841514 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 10497
I20260810 14:54:50.841650 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:54:50.841861 31017 external_mini_cluster.cc:1428] Setting key 97fa27c4770339544f4da4224f262ec6
I20260810 14:54:50.843055 10568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6005b3b0a8204bffa341cb0c9d064536" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 32775 } }
I20260810 14:54:50.843232 10568 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:54:50.843271 10568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6005b3b0a8204bffa341cb0c9d064536, State: Initialized, Role: FOLLOWER
I20260810 14:54:50.843443 10568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6005b3b0a8204bffa341cb0c9d064536" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 32775 } }
I20260810 14:54:50.843564 10568 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:54:50.843629 10568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:54:50.843681 10568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:54:50.844338 10568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6005b3b0a8204bffa341cb0c9d064536" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 32775 } }
I20260810 14:54:50.844446 10568 leader_election.cc:302] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6005b3b0a8204bffa341cb0c9d064536; no voters:
I20260810 14:54:50.844614 10568 leader_election.cc:288] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:54:50.844669 10573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:54:50.844859 10573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [term 1 LEADER]: Becoming Leader. State: Replica: 6005b3b0a8204bffa341cb0c9d064536, State: Running, Role: LEADER
I20260810 14:54:50.844918 10568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:54:50.844995 10573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6005b3b0a8204bffa341cb0c9d064536" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 32775 } }
I20260810 14:54:50.845371 10573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6005b3b0a8204bffa341cb0c9d064536. Latest consensus state: current_term: 1 leader_uuid: "6005b3b0a8204bffa341cb0c9d064536" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6005b3b0a8204bffa341cb0c9d064536" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 32775 } } }
I20260810 14:54:50.845616 10573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [sys.catalog]: This master's current role is: LEADER
I20260810 14:54:50.845542 10575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6005b3b0a8204bffa341cb0c9d064536" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6005b3b0a8204bffa341cb0c9d064536" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 32775 } } }
I20260810 14:54:50.846040 10583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:54:50.846148 10575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536 [sys.catalog]: This master's current role is: LEADER
I20260810 14:54:50.846607 10568 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 14:54:50.846748 10583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 14:54:50 dist-test-slave-0b58 krb5kdc[10027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:54:50.854283 10583 catalog_manager.cc:1347] Generated new cluster ID: d6735246f3bd41f494263e2cdfd32d48
I20260810 14:54:50.854429 10583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:54:50.857368 10572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:50.842424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48712 (local address 127.30.74.126:32775)
0810 14:54:50.842625 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:50.842637 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:50.842681 (+ 44us) server_negotiation.cc:407] Connection header received
0810 14:54:50.843025 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:50.843033 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:50.843099 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:50.843256 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:50.844102 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:50.845616 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:50.846907 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:50.849757 (+ 2850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:50.849996 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:50.853180 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:50.853186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:50.853186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:54:50.853203 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:50.855423 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:50.855808 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:50.855811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:50.855811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:50.855915 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:50.856083 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:50.856084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:50.856084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:50.856399 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:50.856540 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:50.857016 (+ 476us) server_negotiation.cc:299] Negotiation successful
0810 14:54:50.857152 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":82,"threads_started":1}
I20260810 14:54:50.878172 10583 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:54:50.879007 10583 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:54:50.887647 10583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536: Generated new TSK 0
I20260810 14:54:50.887887 10583 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:54:51.591267 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.74.84:32953/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:32775
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:54:51.711208 10617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:54:51.711493 10617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:54:51.711578 10617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:54:51.713011 10617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:54:51.713056 10617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:54:51.713084 10617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:54:51.714709 10617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.74.84:32953/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:32775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:54:51.715022 10617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:54:51.715266 10617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:54:51.720602 10623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:51.720577 10624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:51.720757 10626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:51.721077 10617 server_base.cc:1034] running on GCE node
I20260810 14:54:51.721376 10617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:54:51.721717 10617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:54:51.722918 10617 hybrid_clock.cc:648] HybridClock initialized: now 1786373691722905 us; error 30 us; skew 500 ppm
Aug 10 14:54:51 dist-test-slave-0b58 krb5kdc[10027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373691, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:54:51.725759 10617 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:54:51.726348 10617 webserver.cc:460] Webserver started at http://127.30.74.65:41137/ using document root <none> and password file <none>
I20260810 14:54:51.726629 10617 fs_manager.cc:359] Metadata directory not provided
I20260810 14:54:51.726701 10617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:54:51.726864 10617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:54:51 dist-test-slave-0b58 krb5kdc[10027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373691, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
I20260810 14:54:51.740984 10617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "f51b4a45091045fdaf948ff33012e7d5"
format_stamp: "Formatted at 2026-08-10 14:54:51 on dist-test-slave-0b58"
server_key: "ab8ca7049decc8bdcf553fa0d49bb302"
server_key_iv: "0340fa987e88e87fff1ed5125f5be274"
server_key_version: "kuduclusterkey@0"
I20260810 14:54:51.744117 10617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "f51b4a45091045fdaf948ff33012e7d5"
format_stamp: "Formatted at 2026-08-10 14:54:51 on dist-test-slave-0b58"
server_key: "ab8ca7049decc8bdcf553fa0d49bb302"
server_key_iv: "0340fa987e88e87fff1ed5125f5be274"
server_key_version: "kuduclusterkey@0"
I20260810 14:54:51.750437 10617 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260810 14:54:51.764760 10636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:51.764986 10617 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260810 14:54:51.765107 10617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "f51b4a45091045fdaf948ff33012e7d5"
format_stamp: "Formatted at 2026-08-10 14:54:51 on dist-test-slave-0b58"
server_key: "ab8ca7049decc8bdcf553fa0d49bb302"
server_key_iv: "0340fa987e88e87fff1ed5125f5be274"
server_key_version: "kuduclusterkey@0"
I20260810 14:54:51.765208 10617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:54:51.784689 10617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:54:51.787026 10617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:54:51.787205 10617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:54:51.787485 10617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:54:51.787915 10617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:54:51.787966 10617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:51.787995 10617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:54:51.788012 10617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:51.796556 10617 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:33603
I20260810 14:54:51.796666 10758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:33603 every 8 connection(s)
I20260810 14:54:51.797065 10617 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:54:51.798898 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 10617
I20260810 14:54:51.799023 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:54:51.799228 31017 external_mini_cluster.cc:1428] Setting key 81a68d2eb7c6e297e57f158afeb19928
Aug 10 14:54:51 dist-test-slave-0b58 krb5kdc[10027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373691, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
I20260810 14:54:51.810186 10763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:51.798196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:40563 (local address 127.30.74.126:32775)
0810 14:54:51.798363 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:51.798366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:51.798617 (+ 251us) server_negotiation.cc:407] Connection header received
0810 14:54:51.799392 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:51.799394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:51.799410 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:51.799561 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:51.800469 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:51.801282 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:51.802883 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:51.803193 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:51.803244 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:51.806124 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:51.806125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:51.806126 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:54:51.806136 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:51.808140 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:51.808808 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:51.808810 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:51.808811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:51.808887 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:51.809225 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:51.809226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:51.809226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:51.809460 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:51.809524 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:51.809974 (+ 450us) server_negotiation.cc:299] Negotiation successful
0810 14:54:51.810075 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260810 14:54:51.811064 10762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:51.798141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:32775 (local address 127.30.74.65:40563)
0810 14:54:51.798444 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:51.798498 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:51.799205 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:51.799713 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:51.799718 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:51.800004 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:54:51.800334 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:51.800342 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:51.801415 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:51.801416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:51.802743 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:51.802747 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:51.803351 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:51.803352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:51.803509 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:51.804178 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:54:51.804188 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:54:51.805977 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:54:51.808315 (+ 2338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:54:51.808328 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:54:51.808332 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:54:51.808685 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:54:51.808973 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:54:51.808974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:54:51.808975 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:54:51.809132 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:54:51.809566 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:54:51.809575 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:54:51.809784 (+ 209us) client_negotiation.cc:769] Sending connection context
0810 14:54:51.810603 (+ 819us) client_negotiation.cc:240] Negotiation successful
0810 14:54:51.810818 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":86,"threads_started":1}
I20260810 14:54:51.811959 10759 heartbeater.cc:347] Connected to a master server at 127.30.74.126:32775
I20260810 14:54:51.812085 10759 heartbeater.cc:464] Registering TS with master...
I20260810 14:54:51.812350 10759 heartbeater.cc:511] Master 127.30.74.126:32775 requested a full tablet report, sending...
I20260810 14:54:51.812945 10532 ts_manager.cc:194] Registered new tserver with Master: f51b4a45091045fdaf948ff33012e7d5 (127.30.74.65:33603)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
I20260810 14:54:51.813858 10532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:40563
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:54:51.832474 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.74.84:32953/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:32775
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:54:51.956189 10766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:54:51.956578 10766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:54:51.956688 10766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:54:51.958604 10766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:54:51.958746 10766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:54:51.958849 10766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
Aug 10 14:54:51 dist-test-slave-0b58 krb5kdc[10027](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786373691, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:54:51.961284 10766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.74.84:32953/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:32775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:54:51.961560 10766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:54:51.961920 10766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:54:51.968318 10776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:51.968425 10773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:51.968688 10766 server_base.cc:1034] running on GCE node
W20260810 14:54:51.968851 10774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:51.969111 10766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:54:51.969403 10766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:54:51.970569 10766 hybrid_clock.cc:648] HybridClock initialized: now 1786373691970546 us; error 37 us; skew 500 ppm
Aug 10 14:54:51 dist-test-slave-0b58 krb5kdc[10027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373691, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:54:51.974200 10766 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:54:51.974836 10766 webserver.cc:460] Webserver started at http://127.30.74.66:44037/ using document root <none> and password file <none>
I20260810 14:54:51.975157 10766 fs_manager.cc:359] Metadata directory not provided
I20260810 14:54:51.975277 10766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:54:51.975458 10766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:54:51 dist-test-slave-0b58 krb5kdc[10027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373691, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
I20260810 14:54:52.000512 10766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "340480f64aac4cff8b21f6c69d44a2fc"
format_stamp: "Formatted at 2026-08-10 14:54:51 on dist-test-slave-0b58"
server_key: "90fb24f765b57343c549e99e5fe234cf"
server_key_iv: "77f6fd8efff36e145ce4a28513d9bfe3"
server_key_version: "kuduclusterkey@0"
I20260810 14:54:52.000989 10766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "340480f64aac4cff8b21f6c69d44a2fc"
format_stamp: "Formatted at 2026-08-10 14:54:51 on dist-test-slave-0b58"
server_key: "90fb24f765b57343c549e99e5fe234cf"
server_key_iv: "77f6fd8efff36e145ce4a28513d9bfe3"
server_key_version: "kuduclusterkey@0"
I20260810 14:54:52.003790 10766 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260810 14:54:52.019703 10783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:52.019971 10766 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260810 14:54:52.020079 10766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "340480f64aac4cff8b21f6c69d44a2fc"
format_stamp: "Formatted at 2026-08-10 14:54:51 on dist-test-slave-0b58"
server_key: "90fb24f765b57343c549e99e5fe234cf"
server_key_iv: "77f6fd8efff36e145ce4a28513d9bfe3"
server_key_version: "kuduclusterkey@0"
I20260810 14:54:52.020182 10766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:54:52.035377 10766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:54:52.037719 10766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:54:52.037879 10766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:54:52.038092 10766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:54:52.038466 10766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:54:52.038520 10766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:52.038556 10766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:54:52.038582 10766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:52.046428 10766 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:32897
I20260810 14:54:52.046561 10907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:32897 every 8 connection(s)
I20260810 14:54:52.046890 10766 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:54:52.050894 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 10766
I20260810 14:54:52.051014 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:54:52.051239 31017 external_mini_cluster.cc:1428] Setting key bad10edd4f9f5969ef63c3b475c81ee5
Aug 10 14:54:52 dist-test-slave-0b58 krb5kdc[10027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373691, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:54:52.058786 10763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.047936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:48457 (local address 127.30.74.126:32775)
0810 14:54:52.047993 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:52.047995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:52.048007 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:54:52.048704 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:52.048705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:52.048722 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:52.048797 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:52.049526 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.050347 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.052173 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.052528 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.052571 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:52.055258 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:52.055260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:52.055260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:54:52.055268 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:52.057114 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:52.057629 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:52.057630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:52.057631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:52.057692 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:52.057989 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:52.057990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:52.057990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:52.058121 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:52.058168 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:52.058594 (+ 426us) server_negotiation.cc:299] Negotiation successful
0810 14:54:52.058690 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 14:54:52.059571 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.047648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:32775 (local address 127.30.74.66:48457)
0810 14:54:52.047829 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:52.047879 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:52.048572 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:52.048877 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:52.048882 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:52.049126 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:54:52.049407 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:52.049413 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.050479 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.050479 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:52.052085 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:52.052087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.052663 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.052664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:52.052824 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:52.053456 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:54:52.053466 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:54:52.055150 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:54:52.057228 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:54:52.057236 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:54:52.057238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:54:52.057542 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:54:52.057775 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:54:52.057776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:54:52.057776 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:54:52.057914 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:54:52.058239 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:54:52.058246 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:54:52.058418 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 14:54:52.059131 (+ 713us) client_negotiation.cc:240] Negotiation successful
0810 14:54:52.059327 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260810 14:54:52.060328 10909 heartbeater.cc:347] Connected to a master server at 127.30.74.126:32775
I20260810 14:54:52.060487 10909 heartbeater.cc:464] Registering TS with master...
I20260810 14:54:52.060745 10909 heartbeater.cc:511] Master 127.30.74.126:32775 requested a full tablet report, sending...
I20260810 14:54:52.061204 10532 ts_manager.cc:194] Registered new tserver with Master: 340480f64aac4cff8b21f6c69d44a2fc (127.30.74.66:32897)
I20260810 14:54:52.061682 10532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:48457
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:54:52.090605 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.74.84:32953/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:32775
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:54:52.204602 10916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:54:52.204897 10916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:54:52.204936 10916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:54:52.206681 10916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:54:52.206748 10916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:54:52.206789 10916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:54:52.209084 10916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.74.84:32953/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:32775
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:54:52.209360 10916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:54:52.209707 10916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:54:52.216199 10925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:52.216361 10916 server_base.cc:1034] running on GCE node
W20260810 14:54:52.216471 10922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:54:52.216226 10923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:54:52.216781 10916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:54:52.217048 10916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:54:52.218230 10916 hybrid_clock.cc:648] HybridClock initialized: now 1786373692218200 us; error 39 us; skew 500 ppm
Aug 10 14:54:52 dist-test-slave-0b58 krb5kdc[10027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373692, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:54:52.220906 10916 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:54:52.221357 10916 webserver.cc:460] Webserver started at http://127.30.74.67:41137/ using document root <none> and password file <none>
I20260810 14:54:52.221637 10916 fs_manager.cc:359] Metadata directory not provided
I20260810 14:54:52.221760 10916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:54:52.221997 10916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:54:52 dist-test-slave-0b58 krb5kdc[10027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373692, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
I20260810 14:54:52.237241 10916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "2ed90e9e308946f29b7488a833c9f88d"
format_stamp: "Formatted at 2026-08-10 14:54:52 on dist-test-slave-0b58"
server_key: "abdffafa7957390a9945d7e958a28555"
server_key_iv: "a0da4901a1fd77515c9ef7382441721f"
server_key_version: "kuduclusterkey@0"
I20260810 14:54:52.237704 10916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "2ed90e9e308946f29b7488a833c9f88d"
format_stamp: "Formatted at 2026-08-10 14:54:52 on dist-test-slave-0b58"
server_key: "abdffafa7957390a9945d7e958a28555"
server_key_iv: "a0da4901a1fd77515c9ef7382441721f"
server_key_version: "kuduclusterkey@0"
I20260810 14:54:52.239336 10916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 14:54:52.257210 10932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:52.257452 10916 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:54:52.257555 10916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "2ed90e9e308946f29b7488a833c9f88d"
format_stamp: "Formatted at 2026-08-10 14:54:52 on dist-test-slave-0b58"
server_key: "abdffafa7957390a9945d7e958a28555"
server_key_iv: "a0da4901a1fd77515c9ef7382441721f"
server_key_version: "kuduclusterkey@0"
I20260810 14:54:52.257651 10916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:54:52.271420 10916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:54:52.274260 10916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:54:52.274520 10916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:54:52.274861 10916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:54:52.275322 10916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:54:52.275429 10916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:52.275508 10916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:54:52.275571 10916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:54:52.288837 10916 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:39415
I20260810 14:54:52.289414 10916 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:54:52.290400 11052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:39415 every 8 connection(s)
Aug 10 14:54:52 dist-test-slave-0b58 krb5kdc[10027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373692, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:54:52.299520 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 10916
I20260810 14:54:52.299649 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:54:52.299880 31017 external_mini_cluster.cc:1428] Setting key 81f5d0d0537d1320b36ffdc37288af7f
I20260810 14:54:52.303293 10763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.291536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:52421 (local address 127.30.74.126:32775)
0810 14:54:52.291586 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:52.291588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:52.292112 (+ 524us) server_negotiation.cc:407] Connection header received
0810 14:54:52.292943 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:52.292945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:52.292958 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:52.293046 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:52.293802 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.294553 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.296120 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.296461 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.296509 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:52.299426 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:52.299427 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:52.299427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:54:52.299436 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:52.301561 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:52.302103 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:52.302104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:52.302104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:52.302173 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:52.302490 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:52.302491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:52.302491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:52.302639 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:52.302719 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:52.303103 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 14:54:52.303194 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 14:54:52.303638 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.291230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:32775 (local address 127.30.74.67:52421)
0810 14:54:52.291931 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:52.291977 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:52.292782 (+ 805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:52.293118 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:52.293123 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:52.293388 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:54:52.293676 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:52.293682 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.294677 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.294678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:52.296005 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:52.296008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.296613 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.296614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:52.296788 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:52.297445 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:54:52.297457 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:54:52.299303 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:54:52.301653 (+ 2350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:54:52.301662 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:54:52.301665 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:54:52.302012 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:54:52.302253 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:54:52.302254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:54:52.302255 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:54:52.302410 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:54:52.302726 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:54:52.302732 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:54:52.302938 (+ 206us) client_negotiation.cc:769] Sending connection context
0810 14:54:52.303066 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 14:54:52.303236 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":90,"threads_started":1}
I20260810 14:54:52.304314 11053 heartbeater.cc:347] Connected to a master server at 127.30.74.126:32775
I20260810 14:54:52.304417 11053 heartbeater.cc:464] Registering TS with master...
I20260810 14:54:52.304615 11053 heartbeater.cc:511] Master 127.30.74.126:32775 requested a full tablet report, sending...
I20260810 14:54:52.305056 10532 ts_manager.cc:194] Registered new tserver with Master: 2ed90e9e308946f29b7488a833c9f88d (127.30.74.67:39415)
I20260810 14:54:52.305549 10532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:52421
I20260810 14:54:52.313040 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:54:52.326653 10763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.315013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48718 (local address 127.30.74.126:32775)
0810 14:54:52.315069 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:52.315071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:52.315086 (+ 15us) server_negotiation.cc:407] Connection header received
0810 14:54:52.315167 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:52.315167 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:52.315178 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:52.315248 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:54:52.316022 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.316824 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.317908 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.318203 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.318286 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:52.323249 (+ 4963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:54:52.323251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:54:52.323251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:54:52.323261 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:54:52.325227 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:52.325679 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:52.325680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:52.325681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:52.325744 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:54:52.325953 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:54:52.325954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:54:52.325954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:54:52.326132 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:54:52.326213 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:52.326452 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 14:54:52.326535 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 14:54:52.329618 10532 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:54:52.331998 10532 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:54:52.339850 11068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.334798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:33603 (local address 127.0.0.1:36592)
0810 14:54:52.335097 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:52.335142 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:52.335222 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:52.335588 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:52.335591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:52.335594 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:54:52.335675 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:52.335678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.337045 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.337046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:52.338361 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:52.338364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.339528 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.339529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:52.339625 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:52.339640 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 14:54:52.339712 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 14:54:52.339766 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":173,"threads_started":1}
I20260810 14:54:52.340498 11069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.334914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36592 (local address 127.30.74.65:33603)
0810 14:54:52.335360 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:52.335364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:52.335374 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:54:52.335411 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:52.335414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:52.335430 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:52.335505 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:54:52.335759 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.336938 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.338563 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.339351 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.339428 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:52.339489 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:52.340377 (+ 888us) server_negotiation.cc:299] Negotiation successful
0810 14:54:52.340426 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":71,"threads_started":1}
I20260810 14:54:52.341425 10687 tablet_service.cc:1467] Processing CreateTablet for tablet 86df8eee4f6142ef8d0bfb91ee0725b9 (DEFAULT_TABLE table=test-table [id=3da0b6ce157740d29db4a2e9b4ba548c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:54:52.341832 10687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86df8eee4f6142ef8d0bfb91ee0725b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:54:52.349042 11071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.335137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:32897 (local address 127.0.0.1:33874)
0810 14:54:52.336485 (+ 1348us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:52.336496 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:52.336712 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:52.337190 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:52.337191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:52.337194 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:54:52.338483 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:52.338485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.346386 (+ 7901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.346387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:52.347594 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:52.347597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.348798 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.348799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:52.348874 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:52.348879 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:54:52.348912 (+ 33us) client_negotiation.cc:240] Negotiation successful
0810 14:54:52.348956 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1326,"mutex_wait_us":1248,"thread_start_us":79,"threads_started":1}
I20260810 14:54:52.349678 11072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.335674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33874 (local address 127.30.74.66:32897)
0810 14:54:52.336968 (+ 1294us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:52.336973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:52.336982 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:54:52.337020 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:52.337022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:52.337037 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:52.337110 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:54:52.339307 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.340874 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.347730 (+ 6856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.348679 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.349405 (+ 726us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:52.349508 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:52.349541 (+ 33us) server_negotiation.cc:299] Negotiation successful
0810 14:54:52.349585 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1256,"thread_start_us":97,"threads_started":1}
I20260810 14:54:52.350620 10818 tablet_service.cc:1467] Processing CreateTablet for tablet 86df8eee4f6142ef8d0bfb91ee0725b9 (DEFAULT_TABLE table=test-table [id=3da0b6ce157740d29db4a2e9b4ba548c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:54:52.350845 11070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.335335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:39415 (local address 127.0.0.1:56616)
0810 14:54:52.336802 (+ 1467us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:52.336811 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:52.336850 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:52.337645 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:52.337646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:52.337648 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:54:52.337702 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:52.337704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.347744 (+ 10040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.347745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:52.349194 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:52.349197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.350608 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.350609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:52.350680 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:52.350685 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:54:52.350724 (+ 39us) client_negotiation.cc:240] Negotiation successful
0810 14:54:52.350767 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1451,"thread_start_us":65,"threads_started":1}
I20260810 14:54:52.351027 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86df8eee4f6142ef8d0bfb91ee0725b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:54:52.351256 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.335860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56616 (local address 127.30.74.67:39415)
0810 14:54:52.337283 (+ 1423us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:52.337287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:52.337297 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:54:52.337334 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:52.337336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:52.337351 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:52.337560 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:54:52.337787 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.339203 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.349336 (+ 10133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.350503 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.351013 (+ 510us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:52.351109 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:52.351136 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 14:54:52.351176 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1386,"thread_start_us":197,"threads_started":1}
I20260810 14:54:52.351873 10982 tablet_service.cc:1467] Processing CreateTablet for tablet 86df8eee4f6142ef8d0bfb91ee0725b9 (DEFAULT_TABLE table=test-table [id=3da0b6ce157740d29db4a2e9b4ba548c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:54:52.352183 10982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86df8eee4f6142ef8d0bfb91ee0725b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:54:52.355306 11075 tablet_bootstrap.cc:492] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc: Bootstrap starting.
I20260810 14:54:52.355844 11074 tablet_bootstrap.cc:492] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5: Bootstrap starting.
I20260810 14:54:52.356570 11075 tablet_bootstrap.cc:654] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc: Neither blocks nor log segments found. Creating new log.
I20260810 14:54:52.356679 11074 tablet_bootstrap.cc:654] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5: Neither blocks nor log segments found. Creating new log.
I20260810 14:54:52.357050 11075 log.cc:824] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc: Log is configured to *not* fsync() on all Append() calls
I20260810 14:54:52.357061 11074 log.cc:824] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5: Log is configured to *not* fsync() on all Append() calls
I20260810 14:54:52.357438 11076 tablet_bootstrap.cc:492] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d: Bootstrap starting.
I20260810 14:54:52.358022 11075 tablet_bootstrap.cc:492] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc: No bootstrap required, opened a new log
I20260810 14:54:52.358094 11075 ts_tablet_manager.cc:1397] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 14:54:52.358436 11076 tablet_bootstrap.cc:654] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d: Neither blocks nor log segments found. Creating new log.
I20260810 14:54:52.358754 11076 log.cc:824] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:54:52.359547 11074 tablet_bootstrap.cc:492] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5: No bootstrap required, opened a new log
I20260810 14:54:52.359611 11074 ts_tablet_manager.cc:1397] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260810 14:54:52.360317 11075 raft_consensus.cc:348] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51b4a45091045fdaf948ff33012e7d5" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33603 } } peers { permanent_uuid: "2ed90e9e308946f29b7488a833c9f88d" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39415 } } peers { permanent_uuid: "340480f64aac4cff8b21f6c69d44a2fc" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 32897 } }
I20260810 14:54:52.360492 11075 raft_consensus.cc:374] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:54:52.360527 11075 raft_consensus.cc:729] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 340480f64aac4cff8b21f6c69d44a2fc, State: Initialized, Role: FOLLOWER
I20260810 14:54:52.360652 11075 consensus_queue.cc:260] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51b4a45091045fdaf948ff33012e7d5" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33603 } } peers { permanent_uuid: "2ed90e9e308946f29b7488a833c9f88d" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39415 } } peers { permanent_uuid: "340480f64aac4cff8b21f6c69d44a2fc" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 32897 } }
I20260810 14:54:52.360927 11075 ts_tablet_manager.cc:1428] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:54:52.361395 10909 heartbeater.cc:503] Master 127.30.74.126:32775 was elected leader, sending a full tablet report...
I20260810 14:54:52.363868 11074 raft_consensus.cc:348] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51b4a45091045fdaf948ff33012e7d5" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33603 } } peers { permanent_uuid: "2ed90e9e308946f29b7488a833c9f88d" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39415 } } peers { permanent_uuid: "340480f64aac4cff8b21f6c69d44a2fc" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 32897 } }
I20260810 14:54:52.364060 11074 raft_consensus.cc:374] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:54:52.364095 11074 raft_consensus.cc:729] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f51b4a45091045fdaf948ff33012e7d5, State: Initialized, Role: FOLLOWER
I20260810 14:54:52.364238 11074 consensus_queue.cc:260] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51b4a45091045fdaf948ff33012e7d5" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33603 } } peers { permanent_uuid: "2ed90e9e308946f29b7488a833c9f88d" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39415 } } peers { permanent_uuid: "340480f64aac4cff8b21f6c69d44a2fc" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 32897 } }
I20260810 14:54:52.364594 11074 ts_tablet_manager.cc:1428] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260810 14:54:52.364800 10759 heartbeater.cc:503] Master 127.30.74.126:32775 was elected leader, sending a full tablet report...
I20260810 14:54:52.365477 11076 tablet_bootstrap.cc:492] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d: No bootstrap required, opened a new log
I20260810 14:54:52.365545 11076 ts_tablet_manager.cc:1397] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260810 14:54:52.367955 11076 raft_consensus.cc:348] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51b4a45091045fdaf948ff33012e7d5" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33603 } } peers { permanent_uuid: "2ed90e9e308946f29b7488a833c9f88d" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39415 } } peers { permanent_uuid: "340480f64aac4cff8b21f6c69d44a2fc" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 32897 } }
I20260810 14:54:52.368142 11076 raft_consensus.cc:374] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:54:52.368175 11076 raft_consensus.cc:729] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ed90e9e308946f29b7488a833c9f88d, State: Initialized, Role: FOLLOWER
I20260810 14:54:52.368317 11076 consensus_queue.cc:260] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51b4a45091045fdaf948ff33012e7d5" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33603 } } peers { permanent_uuid: "2ed90e9e308946f29b7488a833c9f88d" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39415 } } peers { permanent_uuid: "340480f64aac4cff8b21f6c69d44a2fc" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 32897 } }
I20260810 14:54:52.368599 11076 ts_tablet_manager.cc:1428] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 14:54:52.368659 11053 heartbeater.cc:503] Master 127.30.74.126:32775 was elected leader, sending a full tablet report...
Aug 10 14:54:52 dist-test-slave-0b58 krb5kdc[10027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373691, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
W20260810 14:54:52.542098 11054 tablet.cc:2369] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:54:52.548120 10760 tablet.cc:2369] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:54:52.548818 10912 tablet.cc:2369] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:54:52.579303 11081 raft_consensus.cc:482] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:54:52.579509 11081 raft_consensus.cc:504] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51b4a45091045fdaf948ff33012e7d5" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33603 } } peers { permanent_uuid: "2ed90e9e308946f29b7488a833c9f88d" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39415 } } peers { permanent_uuid: "340480f64aac4cff8b21f6c69d44a2fc" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 32897 } }
I20260810 14:54:52.579986 11081 leader_election.cc:288] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ed90e9e308946f29b7488a833c9f88d (127.30.74.67:39415), 340480f64aac4cff8b21f6c69d44a2fc (127.30.74.66:32897)
I20260810 14:54:52.584309 11073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.580450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:60395 (local address 127.30.74.67:39415)
0810 14:54:52.580507 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:52.580509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:52.580622 (+ 113us) server_negotiation.cc:407] Connection header received
0810 14:54:52.580755 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:52.580756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:52.580769 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:52.580800 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:54:52.581047 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.581783 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.583303 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.583853 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.583908 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:52.583918 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:52.584154 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 14:54:52.584201 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 14:54:52.584372 11072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.580458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:55937 (local address 127.30.74.66:32897)
0810 14:54:52.580552 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:52.580554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:52.580797 (+ 243us) server_negotiation.cc:407] Connection header received
0810 14:54:52.580922 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:52.580923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:52.580932 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:52.580961 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:54:52.581203 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.581903 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.583411 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.583945 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.583995 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:52.584002 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:52.584240 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 14:54:52.584302 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 14:54:52.584553 11083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.580174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:39415 (local address 127.30.74.65:60395)
0810 14:54:52.580537 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:52.580550 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:52.580658 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:52.580874 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:52.580875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:52.580877 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:54:52.580966 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:52.580968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.581918 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.581919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:52.583186 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:52.583189 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.583989 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.583990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:52.584061 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:52.584066 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:54:52.584405 (+ 339us) client_negotiation.cc:240] Negotiation successful
0810 14:54:52.584458 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":32,"thread_start_us":151,"threads_started":1}
I20260810 14:54:52.584899 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.580216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:32897 (local address 127.30.74.65:55937)
0810 14:54:52.580705 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:52.580716 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:54:52.580846 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:54:52.581022 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:54:52.581023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:54:52.581024 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:54:52.581133 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:52.581135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.582003 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.582004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:52.583314 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:54:52.583317 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.584065 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.584066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:54:52.584156 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:52.584160 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:54:52.584721 (+ 561us) client_negotiation.cc:240] Negotiation successful
0810 14:54:52.584791 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":1,"thread_start_us":119,"threads_started":1}
I20260810 14:54:52.585323 10987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86df8eee4f6142ef8d0bfb91ee0725b9" candidate_uuid: "f51b4a45091045fdaf948ff33012e7d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ed90e9e308946f29b7488a833c9f88d" is_pre_election: true
I20260810 14:54:52.585356 10845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86df8eee4f6142ef8d0bfb91ee0725b9" candidate_uuid: "f51b4a45091045fdaf948ff33012e7d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "340480f64aac4cff8b21f6c69d44a2fc" is_pre_election: true
I20260810 14:54:52.585510 10987 raft_consensus.cc:2454] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f51b4a45091045fdaf948ff33012e7d5 in term 0.
I20260810 14:54:52.585533 10845 raft_consensus.cc:2454] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f51b4a45091045fdaf948ff33012e7d5 in term 0.
I20260810 14:54:52.585821 10637 leader_election.cc:302] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 340480f64aac4cff8b21f6c69d44a2fc, f51b4a45091045fdaf948ff33012e7d5; no voters:
I20260810 14:54:52.585990 11081 raft_consensus.cc:2781] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:54:52.586046 11081 raft_consensus.cc:482] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:54:52.586082 11081 raft_consensus.cc:3037] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:54:52.587150 11081 raft_consensus.cc:504] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51b4a45091045fdaf948ff33012e7d5" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33603 } } peers { permanent_uuid: "2ed90e9e308946f29b7488a833c9f88d" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39415 } } peers { permanent_uuid: "340480f64aac4cff8b21f6c69d44a2fc" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 32897 } }
I20260810 14:54:52.587303 11081 leader_election.cc:288] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [CANDIDATE]: Term 1 election: Requested vote from peers 2ed90e9e308946f29b7488a833c9f88d (127.30.74.67:39415), 340480f64aac4cff8b21f6c69d44a2fc (127.30.74.66:32897)
I20260810 14:54:52.587513 10987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86df8eee4f6142ef8d0bfb91ee0725b9" candidate_uuid: "f51b4a45091045fdaf948ff33012e7d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ed90e9e308946f29b7488a833c9f88d"
I20260810 14:54:52.587560 10845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86df8eee4f6142ef8d0bfb91ee0725b9" candidate_uuid: "f51b4a45091045fdaf948ff33012e7d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "340480f64aac4cff8b21f6c69d44a2fc"
I20260810 14:54:52.587603 10987 raft_consensus.cc:3037] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:54:52.587656 10845 raft_consensus.cc:3037] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:54:52.588661 10987 raft_consensus.cc:2454] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f51b4a45091045fdaf948ff33012e7d5 in term 1.
I20260810 14:54:52.588730 10845 raft_consensus.cc:2454] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f51b4a45091045fdaf948ff33012e7d5 in term 1.
I20260810 14:54:52.588935 10641 leader_election.cc:302] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ed90e9e308946f29b7488a833c9f88d, f51b4a45091045fdaf948ff33012e7d5; no voters:
I20260810 14:54:52.589037 11081 raft_consensus.cc:2781] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:54:52.589355 11081 raft_consensus.cc:686] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [term 1 LEADER]: Becoming Leader. State: Replica: f51b4a45091045fdaf948ff33012e7d5, State: Running, Role: LEADER
I20260810 14:54:52.589462 11081 consensus_queue.cc:237] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51b4a45091045fdaf948ff33012e7d5" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33603 } } peers { permanent_uuid: "2ed90e9e308946f29b7488a833c9f88d" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39415 } } peers { permanent_uuid: "340480f64aac4cff8b21f6c69d44a2fc" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 32897 } }
I20260810 14:54:52.590334 10531 catalog_manager.cc:5495] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to f51b4a45091045fdaf948ff33012e7d5 (127.30.74.65). New cstate: current_term: 1 leader_uuid: "f51b4a45091045fdaf948ff33012e7d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f51b4a45091045fdaf948ff33012e7d5" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ed90e9e308946f29b7488a833c9f88d" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "340480f64aac4cff8b21f6c69d44a2fc" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 32897 } health_report { overall_health: UNKNOWN } } }
I20260810 14:54:52.627424 11069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.624215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36598 (local address 127.30.74.65:33603)
0810 14:54:52.624288 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 14:54:52.624290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:54:52.624298 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:54:52.624341 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:54:52.624342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:54:52.624363 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:54:52.624440 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:54:52.624612 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.625371 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.626455 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:54:52.626756 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:54:52.626837 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:54:52.627028 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:54:52.627161 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:54:52.627244 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 14:54:52.627297 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 14:54:52.627320 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:54:52.629570 10845 raft_consensus.cc:1261] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc [term 1 FOLLOWER]: Refusing update from remote peer f51b4a45091045fdaf948ff33012e7d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:54:52.630050 11081 consensus_queue.cc:1035] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "340480f64aac4cff8b21f6c69d44a2fc" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 32897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:54:52.631343 10987 raft_consensus.cc:1261] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d [term 1 FOLLOWER]: Refusing update from remote peer f51b4a45091045fdaf948ff33012e7d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:54:52.632675 11081 consensus_queue.cc:1035] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ed90e9e308946f29b7488a833c9f88d" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 39415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:54:52.637081 11090 mvcc.cc:205] Tried to move back new op lower bound from 7316986645007863808 to 7316986644846944256. Current Snapshot: MvccSnapshot[applied={T|T < 7316986645007863808}]
I20260810 14:54:52.637221 11089 mvcc.cc:205] Tried to move back new op lower bound from 7316986645007863808 to 7316986644846944256. Current Snapshot: MvccSnapshot[applied={T|T < 7316986645007863808}]
I20260810 14:54:52.637706 11092 mvcc.cc:205] Tried to move back new op lower bound from 7316986645007863808 to 7316986644846944256. Current Snapshot: MvccSnapshot[applied={T|T < 7316986645007863808}]
I20260810 14:54:52.640795 10531 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:54:52.640892 10531 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:54:52.641633 10531 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6005b3b0a8204bffa341cb0c9d064536: Sending DeleteTablet for 3 replicas of tablet 86df8eee4f6142ef8d0bfb91ee0725b9
I20260810 14:54:52.641995 10982 tablet_service.cc:1514] Processing DeleteTablet for tablet 86df8eee4f6142ef8d0bfb91ee0725b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:54:52 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:56616
I20260810 14:54:52.642459 10687 tablet_service.cc:1514] Processing DeleteTablet for tablet 86df8eee4f6142ef8d0bfb91ee0725b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:54:52 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:36592
I20260810 14:54:52.642654 10818 tablet_service.cc:1514] Processing DeleteTablet for tablet 86df8eee4f6142ef8d0bfb91ee0725b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:54:52 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:33874
I20260810 14:54:52.643148 11099 tablet_replica.cc:330] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5: stopping tablet replica
I20260810 14:54:52.643247 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 10617
I20260810 14:54:52.643271 11099 raft_consensus.cc:2229] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:54:52.643396 11099 raft_consensus.cc:2258] T 86df8eee4f6142ef8d0bfb91ee0725b9 P f51b4a45091045fdaf948ff33012e7d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:54:52.643543 11100 tablet_replica.cc:330] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc: stopping tablet replica
I20260810 14:54:52.643623 11100 raft_consensus.cc:2229] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:54:52.643712 11100 raft_consensus.cc:2258] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:54:52.644450 11098 tablet_replica.cc:330] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d: stopping tablet replica
I20260810 14:54:52.644542 11098 raft_consensus.cc:2229] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:54:52.644636 11098 raft_consensus.cc:2258] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:54:52.646042 11098 ts_tablet_manager.cc:1905] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:54:52.648079 11098 ts_tablet_manager.cc:1918] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:54:52.648192 11098 log.cc:1196] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/86df8eee4f6142ef8d0bfb91ee0725b9
I20260810 14:54:52.648826 11100 ts_tablet_manager.cc:1905] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:54:52.650458 11100 ts_tablet_manager.cc:1918] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:54:52.650542 11100 log.cc:1196] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/86df8eee4f6142ef8d0bfb91ee0725b9
I20260810 14:54:52.650825 11100 ts_tablet_manager.cc:1939] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 340480f64aac4cff8b21f6c69d44a2fc: Deleting consensus metadata
I20260810 14:54:52.648883 11098 ts_tablet_manager.cc:1939] T 86df8eee4f6142ef8d0bfb91ee0725b9 P 2ed90e9e308946f29b7488a833c9f88d: Deleting consensus metadata
I20260810 14:54:52.651297 10513 catalog_manager.cc:4841] TS 340480f64aac4cff8b21f6c69d44a2fc (127.30.74.66:32897): tablet 86df8eee4f6142ef8d0bfb91ee0725b9 (table test-table [id=3da0b6ce157740d29db4a2e9b4ba548c]) successfully deleted
I20260810 14:54:52.651386 10516 catalog_manager.cc:4841] TS 2ed90e9e308946f29b7488a833c9f88d (127.30.74.67:39415): tablet 86df8eee4f6142ef8d0bfb91ee0725b9 (table test-table [id=3da0b6ce157740d29db4a2e9b4ba548c]) successfully deleted
W20260810 14:54:52.656860 10513 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:33603 (error 108)
I20260810 14:54:52.657526 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 10766
I20260810 14:54:52.664885 11070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:54:52.664739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:33603 (local address 127.0.0.1:36604)
0810 14:54:52.664807 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 14:54:52.664845 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:33603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 14:54:52.665453 10513 catalog_manager.cc:4584] TS f51b4a45091045fdaf948ff33012e7d5 (127.30.74.65:33603): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 86df8eee4f6142ef8d0bfb91ee0725b9: Network error: Client connection negotiation failed: client connection to 127.30.74.65:33603: connect: Connection refused (error 111)
I20260810 14:54:52.671777 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 10916
I20260810 14:54:52.685216 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 10497
I20260810 14:54:52.692898 31017 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 14:54:53.037734 31017 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 14:54:53.037847 31017 mini_ranger.cc:66] Stopping Ranger...
I20260810 14:54:53.382989 31017 mini_ranger.cc:68] Stopped Ranger
2026-08-10 14:54:53.383 UTC [10057] LOG: received smart shutdown request
2026-08-10 14:54:53.387 UTC [10057] LOG: background worker "logical replication launcher" (PID 10067) exited with exit code 1
2026-08-10 14:54:53.387 UTC [10061] LOG: shutting down
2026-08-10 14:54:53.557 UTC [10057] LOG: database system is shut down
2026-08-10T14:54:53Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49219 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:54:53 dist-test-slave-0b58 krb5kdc[11111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:54:53 dist-test-slave-0b58 krb5kdc[11111](info): set up 2 sockets
Aug 10 14:54:53 dist-test-slave-0b58 krb5kdc[11111](info): commencing operation
krb5kdc: starting...
W20260810 14:54:54.784381 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:54:54 dist-test-slave-0b58 krb5kdc[11111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:54:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:54:54Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.74.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.74.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.74.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/rangeradmin_127.30.74.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.74.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/rangeradmin_127.30.74.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.74.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.74.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.74.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/rangerlookup_127.30.74.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.74.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/rangerlookup_127.30.74.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.74.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.74.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.74.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/HTTP_127.30.74.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.74.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/HTTP_127.30.74.84@KRBTEST.COM.keytab.
I20260810 14:54:54.967048 31017 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task2SWuiv/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/postgres -l logfile start
2026-08-10 14:54:57.618 UTC [11144] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 14:54:57.618 UTC [11144] LOG: listening on IPv4 address "127.30.74.84", port 41181
2026-08-10 14:54:57.621 UTC [11144] LOG: listening on Unix socket "/tmp/.s.PGSQL.41181"
2026-08-10 14:54:57.657 UTC [11147] LOG: database system was shut down at 2026-08-10 14:54:56 UTC
2026-08-10 14:54:57.661 UTC [11144] LOG: database system is ready to accept connections
I20260810 14:54:58.664615 31017 mini_postgres.cc:96] Postgres bound to 41181
2026-08-10 14:54:58.671 UTC [11155] FATAL: database "slave" does not exist
127.30.74.84:41181 - accepting connections
I20260810 14:54:58.672076 31017 mini_ranger.cc:161] Starting Ranger...
I20260810 14:54:58.687999 31017 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 14:54:59.150403 31017 mini_ranger.cc:87] Created ranger Postgres database
I20260810 14:54:59.150552 31017 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-admin
2026-08-10 14:54:59,437 [I] DB FLAVOR :POSTGRES
2026-08-10 14:54:59,438 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 14:54:59,438 [I] Checking connection..
2026-08-10 14:54:59,438 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 14:54:59,703 [I] Checking connection passed.
2026-08-10 14:54:59,703 [I] --------- Verifying version history table ---------
2026-08-10 14:54:59,703 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:54:59,970 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 14:54:59,970 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:55:00,225 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 14:55:00,225 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 14:55:00,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 14:55:00.484 UTC [11223] WARNING: there is no transaction in progress
2026-08-10 14:55:00,492 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 14:55:00,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:55:00,741 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 14:55:00,741 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 14:55:00,742 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 14:55:00,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 14:55:01,231 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-0b58.c.gcp-upstream.internal','N') ;"
2026-08-10 14:55:01,470 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 14:55:01,727 [I] Table x_portal_user does not exist in database ranger
2026-08-10 14:55:01,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 14:55:01,979 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 14:55:01,979 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 14:55:02,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 14:55:02,535 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 14:55:02,535 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 14:55:03.631 UTC [11338] WARNING: there is no transaction in progress
2026-08-10 14:55:03.648 UTC [11338] WARNING: there is no transaction in progress
2026-08-10 14:55:03.667 UTC [11338] WARNING: there is no transaction in progress
2026-08-10 14:55:03.684 UTC [11338] WARNING: there is no transaction in progress
2026-08-10 14:55:03.707 UTC [11338] WARNING: there is no transaction in progress
2026-08-10 14:55:03.730 UTC [11338] WARNING: there is no transaction in progress
2026-08-10 14:55:03.804 UTC [11338] WARNING: there is no transaction in progress
2026-08-10 14:55:03.812 UTC [11338] WARNING: there is no transaction in progress
2026-08-10 14:55:03.829 UTC [11338] WARNING: there is no transaction in progress
2026-08-10 14:55:03.840 UTC [11338] WARNING: there is no transaction in progress
2026-08-10 14:55:03.852 UTC [11338] WARNING: there is no transaction in progress
2026-08-10 14:55:03.868 UTC [11338] WARNING: there is no transaction in progress
2026-08-10 14:55:03.880 UTC [11338] WARNING: there is no transaction in progress
2026-08-10 14:55:03.888 UTC [11338] WARNING: there is no transaction in progress
2026-08-10 14:55:03.895 UTC [11338] WARNING: there is no transaction in progress
2026-08-10 14:55:03.903 UTC [11338] WARNING: there is no transaction in progress
2026-08-10 14:55:04,657 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 14:55:04,658 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 14:55:04,898 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 14:55:04,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 14:55:05,162 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 14:55:05,163 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 14:55:05,405 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 14:55:05,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 14:55:05,680 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 14:55:05,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-0b58.c.gcp-upstream.internal';"
2026-08-10 14:55:06,186 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 14:55:06,187 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 14:55:06,443 [I] DB_PATCHES have already been applied
I20260810 14:55:06.449469 31017 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-admin:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task2SWuiv/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 14:55:06.449551 31017 mini_ranger.cc:193] Using host: 127.30.74.84
I20260810 14:55:06.453637 31017 mini_ranger.cc:227] Ranger admin URL: http://127.30.74.84:48673
Aug 10, 2026 2:55:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 2:55:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 2:55:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 2:55:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 2:55:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 2:55:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 2:55:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 14:55:07.615999 31017 mini_ranger.cc:160] Time spent starting Ranger: real 8.944s user 0.004s sys 0.007s
Aug 10, 2026 2:55:27 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 2:55:27 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 2:55:27 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 2:55:27 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 2:55:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 2:55:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 2:55:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 2:55:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 2:55:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 2:55:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 2:55:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 2:55:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 2:55:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 2:55:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 2:55:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 2:55:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 2:55:27 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 2:55:28 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 14:55:29.226759 31017 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.74.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.74.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.74.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/rangerkms_127.30.74.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.74.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/rangerkms_127.30.74.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.74.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.74.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.74.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/HTTP_127.30.74.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.74.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/HTTP_127.30.74.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 14:55:29 dist-test-slave-0b58 krb5kdc[11111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373729, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 14:55:29.324093 31017 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 14:55:29.335654 31017 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 14:55:30.287760 31017 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 14:55:30.287858 31017 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-kms
I20260810 14:55:30.287880 31017 mini_ranger_kms.cc:227] Using postgres at 127.30.74.84:41181
2026-08-10 14:55:30,460 [I] DB FLAVOR :POSTGRES
2026-08-10 14:55:30,460 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 14:55:30,460 [I] Checking connection
2026-08-10 14:55:30,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 14:55:30,724 [I] connection success
2026-08-10 14:55:30,724 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 14:55:30,724 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 14:55:30,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 14:55:30,995 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 14:55:30,995 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 14:55:30,995 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 14:55:30,996 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.74.84:41181/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 14:55:31,311 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 14:55:31.817390 31017 mini_ranger_kms.cc:310] Created kms service
I20260810 14:55:32.007890 31017 mini_ranger_kms.cc:326] Created kudu user
I20260810 14:55:32.056052 31017 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 14:55:32 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373729, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
I20260810 14:55:32.457096 31017 mini_ranger_kms.cc:384] Added ranger policy
I20260810 14:55:32.457319 31017 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-kms:/tmp/dist-test-task2SWuiv/build/release/bin/postgresql.jar:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task2SWuiv/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 14:55:32.457341 31017 mini_ranger_kms.cc:240] Using host: 127.30.74.84
I20260810 14:55:32.462179 31017 mini_ranger_kms.cc:276] Ranger KMS PID: 11530
I20260810 14:55:32.462285 31017 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.74.84:43505
Aug 10, 2026 2:55:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 2:55:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 2:55:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 2:55:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 2:55:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 2:55:33 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 14:55:34 dist-test-slave-0b58 krb5kdc[11111](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786373734, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.74.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:34.909570 31017 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.585s user 0.004s sys 0.153s
I20260810 14:55:34.909677 31017 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 14:55:34.909722 31017 mini_ranger_kms.cc:401] 127.30.74.84:43505/kms/v1/keys
Aug 10 14:55:35 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373734, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.74.84@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
Aug 10 14:55:35 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373734, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.74.84@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
Aug 10 14:55:37 dist-test-slave-0b58 krb5kdc[11111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:55:37.851122 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:43973
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:43973
--ranger_config_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.74.84:43505/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:55:37.937868 11588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:37.938123 11588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:37.938166 11588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:37.939442 11588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:55:37.939513 11588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:37.939541 11588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:37.939574 11588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:55:37.939607 11588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:55:37.941398 11588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.74.84:43505/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:43973
--ranger_config_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:43973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:37.941704 11588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:37.941943 11588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:37.948390 11594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:37.948383 11593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:37.948426 11588 server_base.cc:1034] running on GCE node
W20260810 14:55:37.948501 11596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:37.948952 11588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:37.949294 11588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:37.950479 11588 hybrid_clock.cc:648] HybridClock initialized: now 1786373737950454 us; error 42 us; skew 500 ppm
Aug 10 14:55:37 dist-test-slave-0b58 krb5kdc[11111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373737, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:37.953207 11588 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:55:37.953768 11588 webserver.cc:460] Webserver started at http://127.30.74.126:45887/ using document root <none> and password file <none>
I20260810 14:55:37.954006 11588 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:37.954046 11588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:37.954190 11588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:55:37 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373737, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
I20260810 14:55:38.003129 11588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "63d9f079cd3949609ed70713d524c113"
format_stamp: "Formatted at 2026-08-10 14:55:37 on dist-test-slave-0b58"
server_key: "ad2d33fbadfe29f92f498f75ba89e2a7"
server_key_iv: "4946d78b858b7677b428355e3b498ba7"
server_key_version: "kuduclusterkey@0"
I20260810 14:55:38.003542 11588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "63d9f079cd3949609ed70713d524c113"
format_stamp: "Formatted at 2026-08-10 14:55:37 on dist-test-slave-0b58"
server_key: "ad2d33fbadfe29f92f498f75ba89e2a7"
server_key_iv: "4946d78b858b7677b428355e3b498ba7"
server_key_version: "kuduclusterkey@0"
I20260810 14:55:38.005185 11588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:55:38.018915 11604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:38.019146 11588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 14:55:38.019273 11588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "63d9f079cd3949609ed70713d524c113"
format_stamp: "Formatted at 2026-08-10 14:55:37 on dist-test-slave-0b58"
server_key: "ad2d33fbadfe29f92f498f75ba89e2a7"
server_key_iv: "4946d78b858b7677b428355e3b498ba7"
server_key_version: "kuduclusterkey@0"
I20260810 14:55:38.019377 11588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:38.036540 11588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:38.038870 11588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:38.039031 11588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:38.044348 11588 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:43973
I20260810 14:55:38.044404 11665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:43973 every 8 connection(s)
I20260810 14:55:38.044925 11588 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:55:38.045715 11666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:55:38.048467 11666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113: Bootstrap starting.
I20260810 14:55:38.048854 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 11588
I20260810 14:55:38.048983 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:55:38.049181 31017 external_mini_cluster.cc:1428] Setting key 870719d187d403d30563a55f90a3c88d
I20260810 14:55:38.049345 11666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113: Neither blocks nor log segments found. Creating new log.
I20260810 14:55:38.049712 11666 log.cc:824] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:38.050730 11666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113: No bootstrap required, opened a new log
I20260810 14:55:38.053217 11666 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d9f079cd3949609ed70713d524c113" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43973 } }
I20260810 14:55:38.053444 11666 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:55:38.053493 11666 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d9f079cd3949609ed70713d524c113, State: Initialized, Role: FOLLOWER
I20260810 14:55:38.053714 11666 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d9f079cd3949609ed70713d524c113" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43973 } }
I20260810 14:55:38.053818 11666 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:55:38.053853 11666 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:55:38.053932 11666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:55:38.054798 11666 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d9f079cd3949609ed70713d524c113" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43973 } }
I20260810 14:55:38.054920 11666 leader_election.cc:302] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63d9f079cd3949609ed70713d524c113; no voters:
I20260810 14:55:38.055179 11666 leader_election.cc:288] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:55:38.055244 11673 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:55:38.055492 11673 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 1 LEADER]: Becoming Leader. State: Replica: 63d9f079cd3949609ed70713d524c113, State: Running, Role: LEADER
I20260810 14:55:38.055619 11666 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:55:38.055636 11673 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d9f079cd3949609ed70713d524c113" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43973 } }
I20260810 14:55:38.056131 11673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63d9f079cd3949609ed70713d524c113" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d9f079cd3949609ed70713d524c113" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43973 } } }
I20260810 14:55:38.056324 11673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [sys.catalog]: This master's current role is: LEADER
I20260810 14:55:38.056509 11673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63d9f079cd3949609ed70713d524c113. Latest consensus state: current_term: 1 leader_uuid: "63d9f079cd3949609ed70713d524c113" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d9f079cd3949609ed70713d524c113" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43973 } } }
I20260810 14:55:38.056604 11673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [sys.catalog]: This master's current role is: LEADER
I20260810 14:55:38.056809 11681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:55:38.057375 11681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:55:38.057740 11666 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 10 14:55:38 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:38.061949 11693 mvcc.cc:205] Tried to move back new op lower bound from 7316986831087312896 to 7316986831076880384. Current Snapshot: MvccSnapshot[applied={T|T < 7316986831087312896}]
I20260810 14:55:38.064248 11681 catalog_manager.cc:1347] Generated new cluster ID: cf1fe23437cf42ca862be55741ae1dbf
I20260810 14:55:38.064373 11681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:55:38.067200 11670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:38.049780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49756 (local address 127.30.74.126:43973)
0810 14:55:38.049993 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:38.050003 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:38.050041 (+ 38us) server_negotiation.cc:407] Connection header received
0810 14:55:38.050652 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:38.050662 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:38.050742 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:38.050965 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:38.051826 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:38.053365 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:38.054567 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:38.055264 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:38.058640 (+ 3376us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:38.058770 (+ 130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:38.058775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:38.058775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:38.058787 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:38.064241 (+ 5454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:38.064853 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:38.064857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:38.064858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:38.065013 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:38.065275 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:38.065276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:38.065276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:38.065738 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:38.066406 (+ 668us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:38.066498 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 14:55:38.066866 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":87,"threads_started":1}
I20260810 14:55:38.104962 11681 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:55:38.105836 11681 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:55:38.119686 11681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113: Generated new TSK 0
I20260810 14:55:38.120033 11681 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:55:38.884507 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.74.84:43505/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43973
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:55:39.014650 11718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:39.015007 11718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:39.015053 11718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:39.017025 11718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:39.017105 11718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:39.017150 11718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:55:39.019654 11718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.74.84:43505/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:39.019961 11718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:39.020313 11718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:39.026480 11724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:39.026480 11726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:39.026485 11723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:39.026834 11718 server_base.cc:1034] running on GCE node
I20260810 14:55:39.027065 11718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:39.027369 11718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:39.028704 11718 hybrid_clock.cc:648] HybridClock initialized: now 1786373739028680 us; error 33 us; skew 500 ppm
Aug 10 14:55:39 dist-test-slave-0b58 krb5kdc[11111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373739, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:39.031692 11718 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:55:39.032223 11718 webserver.cc:460] Webserver started at http://127.30.74.65:44383/ using document root <none> and password file <none>
I20260810 14:55:39.032517 11718 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:39.032596 11718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:39.032766 11718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:55:39 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373739, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
I20260810 14:55:39.046123 11718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "5b0e4d8276b24b33ac8c9d150c97054a"
format_stamp: "Formatted at 2026-08-10 14:55:39 on dist-test-slave-0b58"
server_key: "5cc1c8de95baa8878fc7d5c52ac2ec9d"
server_key_iv: "eaa62cee61f09c2472ea2ce99ddc7699"
server_key_version: "kuduclusterkey@0"
I20260810 14:55:39.046803 11718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "5b0e4d8276b24b33ac8c9d150c97054a"
format_stamp: "Formatted at 2026-08-10 14:55:39 on dist-test-slave-0b58"
server_key: "5cc1c8de95baa8878fc7d5c52ac2ec9d"
server_key_iv: "eaa62cee61f09c2472ea2ce99ddc7699"
server_key_version: "kuduclusterkey@0"
I20260810 14:55:39.049474 11718 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260810 14:55:39.068547 11735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:39.068847 11718 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.001s
I20260810 14:55:39.068962 11718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "5b0e4d8276b24b33ac8c9d150c97054a"
format_stamp: "Formatted at 2026-08-10 14:55:39 on dist-test-slave-0b58"
server_key: "5cc1c8de95baa8878fc7d5c52ac2ec9d"
server_key_iv: "eaa62cee61f09c2472ea2ce99ddc7699"
server_key_version: "kuduclusterkey@0"
I20260810 14:55:39.069080 11718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:39.088778 11718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:39.091784 11718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:39.092052 11718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:39.092437 11718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:55:39.092905 11718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:55:39.093003 11718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:39.093127 11718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:55:39.093209 11718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:39.101094 11718 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:44301
I20260810 14:55:39.101881 11718 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:55:39.103657 11857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:44301 every 8 connection(s)
I20260810 14:55:39.108807 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 11718
I20260810 14:55:39.108930 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:55:39.109128 31017 external_mini_cluster.cc:1428] Setting key 76ebe2f4bf9082ada5edffef00e8c6b7
Aug 10 14:55:39 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373739, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:39.119295 11861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:39.104454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43973 (local address 127.30.74.65:58697)
0810 14:55:39.104895 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:39.104951 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:39.105622 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:39.105927 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:39.105931 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:39.106169 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:39.106389 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:39.106394 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:39.107143 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:39.107145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:39.108473 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:39.108477 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:39.112545 (+ 4068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:39.112546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:39.112748 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:39.113441 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:39.113453 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:39.115237 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:39.117463 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:39.117473 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:39.117476 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:39.117829 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:39.118058 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:39.118059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:39.118059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:39.118199 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:39.118499 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:39.118506 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:39.118706 (+ 200us) client_negotiation.cc:769] Sending connection context
0810 14:55:39.118825 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 14:55:39.119005 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":102,"threads_started":1}
I20260810 14:55:39.119405 11860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:39.104464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:58697 (local address 127.30.74.126:43973)
0810 14:55:39.104795 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:39.104798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:39.105568 (+ 770us) server_negotiation.cc:407] Connection header received
0810 14:55:39.105748 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:39.105749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:39.105766 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:39.105849 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:39.106510 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:39.107036 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:39.111950 (+ 4914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:39.112353 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:39.112433 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:39.115368 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:39.115369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:39.115370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:39.115379 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:39.117351 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:39.117922 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:39.117923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:39.117924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:39.117991 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:39.118269 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:39.118270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:39.118270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:39.118416 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:39.118473 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:39.119192 (+ 719us) server_negotiation.cc:299] Negotiation successful
0810 14:55:39.119294 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":82,"threads_started":1}
I20260810 14:55:39.120419 11858 heartbeater.cc:347] Connected to a master server at 127.30.74.126:43973
I20260810 14:55:39.120532 11858 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:39.120770 11858 heartbeater.cc:511] Master 127.30.74.126:43973 requested a full tablet report, sending...
I20260810 14:55:39.121359 11624 ts_manager.cc:194] Registered new tserver with Master: 5b0e4d8276b24b33ac8c9d150c97054a (127.30.74.65:44301)
I20260810 14:55:39.122298 11624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:58697
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:55:39.151558 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.74.84:43505/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43973
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:55:39.269795 11864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:39.270061 11864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:39.270109 11864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:39.271561 11864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:39.271649 11864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:39.271701 11864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:55:39.273484 11864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.74.84:43505/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:39.273953 11864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:39.274312 11864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:39.280383 11871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:39.280445 11875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:39.280687 11873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:39.280946 11864 server_base.cc:1034] running on GCE node
I20260810 14:55:39.281184 11864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:39.281503 11864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:39.282673 11864 hybrid_clock.cc:648] HybridClock initialized: now 1786373739282662 us; error 35 us; skew 500 ppm
Aug 10 14:55:39 dist-test-slave-0b58 krb5kdc[11111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373739, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:39.285629 11864 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:55:39.286170 11864 webserver.cc:460] Webserver started at http://127.30.74.66:35467/ using document root <none> and password file <none>
I20260810 14:55:39.286501 11864 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:39.286567 11864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:39.286696 11864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:55:39 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373739, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
I20260810 14:55:39.301096 11864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "0889ab1831c84c3d91a71dbd12e4cac5"
format_stamp: "Formatted at 2026-08-10 14:55:39 on dist-test-slave-0b58"
server_key: "23d7a045195feda3927a498bc26ba9e7"
server_key_iv: "26a083545246131f140831c8af6fc17d"
server_key_version: "kuduclusterkey@0"
I20260810 14:55:39.301615 11864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "0889ab1831c84c3d91a71dbd12e4cac5"
format_stamp: "Formatted at 2026-08-10 14:55:39 on dist-test-slave-0b58"
server_key: "23d7a045195feda3927a498bc26ba9e7"
server_key_iv: "26a083545246131f140831c8af6fc17d"
server_key_version: "kuduclusterkey@0"
I20260810 14:55:39.305246 11864 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
Aug 10 14:55:39 dist-test-slave-0b58 krb5kdc[11111](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786373739, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:39.321689 11882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:39.321883 11864 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260810 14:55:39.321985 11864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "0889ab1831c84c3d91a71dbd12e4cac5"
format_stamp: "Formatted at 2026-08-10 14:55:39 on dist-test-slave-0b58"
server_key: "23d7a045195feda3927a498bc26ba9e7"
server_key_iv: "26a083545246131f140831c8af6fc17d"
server_key_version: "kuduclusterkey@0"
I20260810 14:55:39.322088 11864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:39.344343 11864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:39.354314 11864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:39.354526 11864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:39.354786 11864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:55:39.355250 11864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:55:39.355307 11864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:39.355341 11864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:55:39.355360 11864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:39.363081 11864 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:33215
I20260810 14:55:39.363628 11864 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:55:39.365510 12004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:33215 every 8 connection(s)
I20260810 14:55:39.370553 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 11864
I20260810 14:55:39.370687 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:55:39.370920 31017 external_mini_cluster.cc:1428] Setting key 09fd8a6f3375c789b85063a1e84183cd
Aug 10 14:55:39 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373739, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:39.379305 12007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:39.366806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43973 (local address 127.30.74.66:50893)
0810 14:55:39.367424 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:39.367474 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:39.368216 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:39.368674 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:39.368679 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:39.368966 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:39.369282 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:39.369290 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:39.370397 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:39.370399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:39.371743 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:39.371747 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:39.372400 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:39.372400 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:39.372552 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:39.373289 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:39.373301 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:39.375385 (+ 2084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:39.377505 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:39.377516 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:39.377520 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:39.377857 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:39.378091 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:39.378092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:39.378093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:39.378234 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:39.378560 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:39.378566 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:39.378748 (+ 182us) client_negotiation.cc:769] Sending connection context
0810 14:55:39.378861 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 14:55:39.379033 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":111,"threads_started":1}
I20260810 14:55:39.380076 11860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:39.367139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:50893 (local address 127.30.74.126:43973)
0810 14:55:39.367216 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:39.367219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:39.367592 (+ 373us) server_negotiation.cc:407] Connection header received
0810 14:55:39.368407 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:39.368408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:39.368428 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:39.368462 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:39.369410 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:39.370173 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:39.371864 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:39.372183 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:39.372249 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:39.375529 (+ 3280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:39.375532 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:39.375533 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:39.375542 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:39.377386 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:39.377955 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:39.377956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:39.377957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:39.378024 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:39.378320 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:39.378321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:39.378321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:39.378475 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:39.378533 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:39.379877 (+ 1344us) server_negotiation.cc:299] Negotiation successful
0810 14:55:39.379981 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 14:55:39.380541 12005 heartbeater.cc:347] Connected to a master server at 127.30.74.126:43973
I20260810 14:55:39.380656 12005 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:39.380875 12005 heartbeater.cc:511] Master 127.30.74.126:43973 requested a full tablet report, sending...
I20260810 14:55:39.381364 11624 ts_manager.cc:194] Registered new tserver with Master: 0889ab1831c84c3d91a71dbd12e4cac5 (127.30.74.66:33215)
I20260810 14:55:39.381877 11624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:50893
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:55:39.409401 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.74.84:43505/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43973
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:55:39.534969 12010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:39.535393 12010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:39.535472 12010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:39.537340 12010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:39.537431 12010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:39.537479 12010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:55:39.539247 12010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.74.84:43505/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:39.539577 12010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:39.539834 12010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:39.546442 12020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:39.546551 12017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:39.546607 12018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:39.547756 12010 server_base.cc:1034] running on GCE node
I20260810 14:55:39.548020 12010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:39.548388 12010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:39.549579 12010 hybrid_clock.cc:648] HybridClock initialized: now 1786373739549553 us; error 40 us; skew 500 ppm
Aug 10 14:55:39 dist-test-slave-0b58 krb5kdc[11111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373739, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:39.553251 12010 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:55:39.553998 12010 webserver.cc:460] Webserver started at http://127.30.74.67:42477/ using document root <none> and password file <none>
I20260810 14:55:39.554334 12010 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:39.554479 12010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:39.554682 12010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:55:39 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373739, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
I20260810 14:55:39.571283 12010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "8c945aa6d7b24b0a912f6b45358b21f7"
format_stamp: "Formatted at 2026-08-10 14:55:39 on dist-test-slave-0b58"
server_key: "ba51ea4f057cfd55efa55febbcbb0380"
server_key_iv: "35ae8d9831a23f909425ad5b94b656fb"
server_key_version: "kuduclusterkey@0"
I20260810 14:55:39.571957 12010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "8c945aa6d7b24b0a912f6b45358b21f7"
format_stamp: "Formatted at 2026-08-10 14:55:39 on dist-test-slave-0b58"
server_key: "ba51ea4f057cfd55efa55febbcbb0380"
server_key_iv: "35ae8d9831a23f909425ad5b94b656fb"
server_key_version: "kuduclusterkey@0"
I20260810 14:55:39.583889 12010 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.004s sys 0.000s
I20260810 14:55:39.599346 12027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:39.599592 12010 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 14:55:39.599726 12010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "8c945aa6d7b24b0a912f6b45358b21f7"
format_stamp: "Formatted at 2026-08-10 14:55:39 on dist-test-slave-0b58"
server_key: "ba51ea4f057cfd55efa55febbcbb0380"
server_key_iv: "35ae8d9831a23f909425ad5b94b656fb"
server_key_version: "kuduclusterkey@0"
I20260810 14:55:39.599862 12010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:39.614085 12010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:39.616493 12010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:39.616677 12010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:39.616948 12010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:55:39.617327 12010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:55:39.617486 12010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:39.617542 12010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:55:39.617571 12010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:39.625547 12010 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:36413
I20260810 14:55:39.626135 12010 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:55:39.635648 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 12010
I20260810 14:55:39.635846 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:55:39.636053 31017 external_mini_cluster.cc:1428] Setting key 907bc0652f56d77fc58f75c1969129aa
I20260810 14:55:39.637521 12145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:36413 every 8 connection(s)
Aug 10 14:55:39 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373739, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:39.655937 11860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:39.637152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:44115 (local address 127.30.74.126:43973)
0810 14:55:39.637225 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:39.637227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:39.644362 (+ 7135us) server_negotiation.cc:407] Connection header received
0810 14:55:39.645232 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:39.645233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:39.645250 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:39.645286 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:39.646367 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:39.647150 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:39.648865 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:39.649185 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:39.649233 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:39.652217 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:39.652219 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:39.652219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:39.652228 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:39.654091 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:39.654691 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:39.654693 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:39.654693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:39.654761 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:39.655100 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:39.655101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:39.655101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:39.655251 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:39.655301 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:39.655729 (+ 428us) server_negotiation.cc:299] Negotiation successful
0810 14:55:39.655837 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 14:55:39.656523 12148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:39.637182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43973 (local address 127.30.74.67:44115)
0810 14:55:39.644102 (+ 6920us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:39.644192 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:39.645075 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:39.645560 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:39.645565 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:39.645886 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:39.646237 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:39.646244 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:39.647296 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:39.647297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:39.648730 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:39.648734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:39.649355 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:39.649356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:39.649514 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:39.650249 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:39.650261 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:39.652083 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:39.654211 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:39.654221 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:39.654224 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:39.654586 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:39.654858 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:39.654858 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:39.654859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:39.655017 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:39.655359 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:39.655366 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:39.655556 (+ 190us) client_negotiation.cc:769] Sending connection context
0810 14:55:39.656052 (+ 496us) client_negotiation.cc:240] Negotiation successful
0810 14:55:39.656246 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6843,"thread_start_us":126,"threads_started":1}
I20260810 14:55:39.657656 12146 heartbeater.cc:347] Connected to a master server at 127.30.74.126:43973
I20260810 14:55:39.657783 12146 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:39.658047 12146 heartbeater.cc:511] Master 127.30.74.126:43973 requested a full tablet report, sending...
I20260810 14:55:39.659087 11624 ts_manager.cc:194] Registered new tserver with Master: 8c945aa6d7b24b0a912f6b45358b21f7 (127.30.74.67:36413)
I20260810 14:55:39.659592 11624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:44115
I20260810 14:55:39.670063 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:55:39.670989 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 11718
I20260810 14:55:39.690080 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 11864
I20260810 14:55:39.703493 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 12010
I20260810 14:55:39.711279 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 11588
I20260810 14:55:39.724591 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:43973
--webserver_interface=127.30.74.126
--webserver_port=45887
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:43973
--ranger_config_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.74.84:43505/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.74.84:43505/kms with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:55:39.809202 12149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:39.809496 12149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:39.809537 12149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:39.810822 12149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:55:39.810902 12149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:39.810927 12149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:39.810947 12149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:55:39.810981 12149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:55:39.812762 12149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.74.84:43505/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:43973
--ranger_config_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:43973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=45887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:39.813097 12149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:39.813355 12149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:39.818639 12159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:39.818642 12154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:39.818639 12156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:39.819154 12149 server_base.cc:1034] running on GCE node
I20260810 14:55:39.819376 12149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:39.819648 12149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:39.820832 12149 hybrid_clock.cc:648] HybridClock initialized: now 1786373739820794 us; error 46 us; skew 500 ppm
Aug 10 14:55:39 dist-test-slave-0b58 krb5kdc[11111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373739, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:39.823675 12149 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:55:39.824128 12149 webserver.cc:460] Webserver started at http://127.30.74.126:45887/ using document root <none> and password file <none>
I20260810 14:55:39.824434 12149 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:39.824503 12149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 14:55:39 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373739, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
I20260810 14:55:39.912858 12149 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:55:39.913918 12167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:39.914160 12149 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:39.914278 12149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "63d9f079cd3949609ed70713d524c113"
format_stamp: "Formatted at 2026-08-10 14:55:37 on dist-test-slave-0b58"
server_key: "ad2d33fbadfe29f92f498f75ba89e2a7"
server_key_iv: "4946d78b858b7677b428355e3b498ba7"
server_key_version: "kuduclusterkey@0"
I20260810 14:55:39.914801 12149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:39.942728 12149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:39.944813 12149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:39.944983 12149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:39.949568 12149 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:43973
I20260810 14:55:39.949609 12221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:43973 every 8 connection(s)
I20260810 14:55:39.950037 12149 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:55:39.951452 12223 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:55:39.952131 12223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113: Bootstrap starting.
I20260810 14:55:39.952529 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 12149
I20260810 14:55:39.952999 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.74.84:43505/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:44301
--local_ip_for_outbound_sockets=127.30.74.65
--tserver_master_addrs=127.30.74.126:43973
--webserver_port=44383
--webserver_interface=127.30.74.65
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.74.84:43505/kms with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:55:39.962289 12223 log.cc:824] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:39.964491 12223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:55:39.964816 12223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113: Bootstrap complete.
I20260810 14:55:39.967407 12223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d9f079cd3949609ed70713d524c113" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43973 } }
I20260810 14:55:39.967700 12223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d9f079cd3949609ed70713d524c113, State: Initialized, Role: FOLLOWER
I20260810 14:55:39.967921 12223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d9f079cd3949609ed70713d524c113" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43973 } }
I20260810 14:55:39.968008 12223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:55:39.968066 12223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:55:39.968117 12223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:55:39.969312 12223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d9f079cd3949609ed70713d524c113" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43973 } }
I20260810 14:55:39.969453 12223 leader_election.cc:302] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63d9f079cd3949609ed70713d524c113; no voters:
I20260810 14:55:39.969681 12223 leader_election.cc:288] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 14:55:39.969780 12228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 14:55:39.969951 12228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [term 2 LEADER]: Becoming Leader. State: Replica: 63d9f079cd3949609ed70713d524c113, State: Running, Role: LEADER
I20260810 14:55:39.969977 12223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:55:39.970065 12228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d9f079cd3949609ed70713d524c113" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43973 } }
I20260810 14:55:39.970391 12228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63d9f079cd3949609ed70713d524c113. Latest consensus state: current_term: 2 leader_uuid: "63d9f079cd3949609ed70713d524c113" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d9f079cd3949609ed70713d524c113" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43973 } } }
I20260810 14:55:39.970566 12228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [sys.catalog]: This master's current role is: LEADER
I20260810 14:55:39.970388 12229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "63d9f079cd3949609ed70713d524c113" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63d9f079cd3949609ed70713d524c113" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43973 } } }
I20260810 14:55:39.970978 12229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113 [sys.catalog]: This master's current role is: LEADER
I20260810 14:55:39.971305 12236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:55:39.971695 12223 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 14:55:39.971861 12236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:55:39.971995 12236 catalog_manager.cc:1259] Loaded cluster ID: cf1fe23437cf42ca862be55741ae1dbf
I20260810 14:55:39.972034 12236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:55:39.976449 12236 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:55:39.977133 12236 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113: Loaded TSK: 0
I20260810 14:55:39.977440 12236 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 14:55:40.083242 12226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:40.083593 12226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:40.083637 12226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:40.085915 12226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:40.086006 12226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:40.086054 12226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:55:40.089087 12226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.74.84:43505/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:44301
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=44383
--tserver_master_addrs=127.30.74.126:43973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:40.089419 12226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:40.089752 12226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:40.096884 12265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:40.097539 12268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:40.098265 12266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:40.098479 12226 server_base.cc:1034] running on GCE node
I20260810 14:55:40.098762 12226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:40.099129 12226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:40.100350 12226 hybrid_clock.cc:648] HybridClock initialized: now 1786373740100313 us; error 48 us; skew 500 ppm
Aug 10 14:55:40 dist-test-slave-0b58 krb5kdc[11111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373740, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:40.105607 12226 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:55:40.106410 12226 webserver.cc:460] Webserver started at http://127.30.74.65:44383/ using document root <none> and password file <none>
I20260810 14:55:40.106760 12226 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:40.106866 12226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 14:55:40 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373740, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
I20260810 14:55:40.140758 12226 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.001s sys 0.000s
I20260810 14:55:40.141971 12277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:40.142213 12226 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:40.142324 12226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "5b0e4d8276b24b33ac8c9d150c97054a"
format_stamp: "Formatted at 2026-08-10 14:55:39 on dist-test-slave-0b58"
server_key: "5cc1c8de95baa8878fc7d5c52ac2ec9d"
server_key_iv: "eaa62cee61f09c2472ea2ce99ddc7699"
server_key_version: "kuduclusterkey@0"
I20260810 14:55:40.142771 12226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:40.156876 12226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:40.159786 12226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:40.159991 12226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:40.160343 12226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:55:40.160794 12226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:55:40.160854 12226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:40.160898 12226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:55:40.160924 12226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:40.167840 12226 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:44301
I20260810 14:55:40.168125 12399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:44301 every 8 connection(s)
I20260810 14:55:40.168414 12226 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:55:40.172808 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 12226
I20260810 14:55:40.173151 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.74.84:43505/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:33215
--local_ip_for_outbound_sockets=127.30.74.66
--tserver_master_addrs=127.30.74.126:43973
--webserver_port=35467
--webserver_interface=127.30.74.66
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.74.84:43505/kms with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:55:40 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373740, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:40.185810 12402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:40.169351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43973 (local address 127.30.74.65:55975)
0810 14:55:40.169585 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:40.169631 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:40.170261 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:40.174516 (+ 4255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:40.174522 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:40.174820 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:40.175146 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:40.175155 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:40.177025 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:40.177027 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:40.177885 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:40.177889 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:40.178615 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:40.178616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:40.178774 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:40.179707 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:40.179717 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:40.181412 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:40.183594 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:40.183603 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:40.183606 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:40.183870 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:40.184180 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:40.184181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:40.184181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:40.184387 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:40.184971 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:40.184978 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:40.185190 (+ 212us) client_negotiation.cc:769] Sending connection context
0810 14:55:40.185307 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 14:55:40.185496 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":96,"threads_started":1}
I20260810 14:55:40.186803 12403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:40.169444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:55975 (local address 127.30.74.126:43973)
0810 14:55:40.173853 (+ 4409us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:40.173866 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:40.173928 (+ 62us) server_negotiation.cc:407] Connection header received
0810 14:55:40.174137 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:40.174145 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:40.174227 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:40.174415 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:40.175349 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:40.176884 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:40.178017 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:40.178480 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:40.178619 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:40.181587 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:40.181594 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:40.181594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:40.181617 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:40.183470 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:40.183970 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:40.183975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:40.183975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:40.184084 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:40.184476 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:40.184477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:40.184477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:40.184849 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:40.184937 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:40.186359 (+ 1422us) server_negotiation.cc:299] Negotiation successful
0810 14:55:40.186517 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4321,"thread_start_us":96,"threads_started":1}
I20260810 14:55:40.188238 12400 heartbeater.cc:347] Connected to a master server at 127.30.74.126:43973
I20260810 14:55:40.188385 12400 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:40.188621 12400 heartbeater.cc:511] Master 127.30.74.126:43973 requested a full tablet report, sending...
W20260810 14:55:40.189764 12400 heartbeater.cc:644] Failed to heartbeat to 127.30.74.126:43973 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 14:55:40.189844 12400 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:40.189925 12400 heartbeater.cc:511] Master 127.30.74.126:43973 requested a full tablet report, sending...
I20260810 14:55:40.190416 12400 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:40.190505 12400 heartbeater.cc:511] Master 127.30.74.126:43973 requested a full tablet report, sending...
W20260810 14:55:40.191162 12400 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:55:40.319336 12404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:40.319633 12404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:40.319684 12404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:40.321678 12404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:40.321753 12404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:40.321801 12404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:55:40.324200 12404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.74.84:43505/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:33215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=35467
--tserver_master_addrs=127.30.74.126:43973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:40.324513 12404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:40.324838 12404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:40.333832 12410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:40.334169 12414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:40.334492 12411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:40.343819 12404 server_base.cc:1034] running on GCE node
I20260810 14:55:40.344146 12404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:40.344506 12404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:40.347820 12404 hybrid_clock.cc:648] HybridClock initialized: now 1786373740347780 us; error 47 us; skew 500 ppm
Aug 10 14:55:40 dist-test-slave-0b58 krb5kdc[11111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373740, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:40.351387 12404 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:55:40.351994 12404 webserver.cc:460] Webserver started at http://127.30.74.66:35467/ using document root <none> and password file <none>
I20260810 14:55:40.352321 12404 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:40.352406 12404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 14:55:40 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373740, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
I20260810 14:55:40.369187 12404 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260810 14:55:40.380441 12423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:40.380769 12404 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.000s sys 0.001s
I20260810 14:55:40.380955 12404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "0889ab1831c84c3d91a71dbd12e4cac5"
format_stamp: "Formatted at 2026-08-10 14:55:39 on dist-test-slave-0b58"
server_key: "23d7a045195feda3927a498bc26ba9e7"
server_key_iv: "26a083545246131f140831c8af6fc17d"
server_key_version: "kuduclusterkey@0"
I20260810 14:55:40.381721 12404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:40.395327 12404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:40.398077 12404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:40.398347 12404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:40.398670 12404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:55:40.399070 12404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:55:40.399121 12404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:40.399168 12404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:55:40.399195 12404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:40.406243 12404 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:33215
I20260810 14:55:40.406795 12404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:55:40.410439 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 12404
I20260810 14:55:40.410780 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.74.84:43505/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:36413
--local_ip_for_outbound_sockets=127.30.74.67
--tserver_master_addrs=127.30.74.126:43973
--webserver_port=42477
--webserver_interface=127.30.74.67
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.74.84:43505/kms with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:55:40.417285 12543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:33215 every 8 connection(s)
Aug 10 14:55:40 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373740, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:40.431947 12403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:40.418338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:38767 (local address 127.30.74.126:43973)
0810 14:55:40.418391 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:40.418393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:40.419211 (+ 818us) server_negotiation.cc:407] Connection header received
0810 14:55:40.420085 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:40.420086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:40.420106 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:40.420155 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:40.421157 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:40.421996 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:40.423756 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:40.424113 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:40.424179 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:40.427499 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:40.427501 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:40.427501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:40.427510 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:40.429365 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:40.430462 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:40.430464 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:40.430465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:40.430532 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:40.430820 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:40.430821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:40.430821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:40.430961 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:40.431002 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:40.431732 (+ 730us) server_negotiation.cc:299] Negotiation successful
0810 14:55:40.431841 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 14:55:40.438119 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:40.418036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43973 (local address 127.30.74.66:38767)
0810 14:55:40.419016 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:40.419080 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:40.419937 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:40.420411 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:40.420416 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:40.420686 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:40.421011 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:40.421019 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:40.422152 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:40.422153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:40.423635 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:40.423638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:40.424340 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:40.424341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:40.424509 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:40.425387 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:40.425398 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:40.427372 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:40.429469 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:40.429479 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:40.429482 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:40.430356 (+ 874us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:40.430603 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:40.430603 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:40.430604 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:40.430747 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:40.431047 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:40.431052 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:40.431555 (+ 503us) client_negotiation.cc:769] Sending connection context
0810 14:55:40.432437 (+ 882us) client_negotiation.cc:240] Negotiation successful
0810 14:55:40.432627 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"thread_start_us":99,"threads_started":1}
I20260810 14:55:40.438752 12544 heartbeater.cc:347] Connected to a master server at 127.30.74.126:43973
I20260810 14:55:40.438892 12544 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:40.439141 12544 heartbeater.cc:511] Master 127.30.74.126:43973 requested a full tablet report, sending...
W20260810 14:55:40.439893 12544 heartbeater.cc:644] Failed to heartbeat to 127.30.74.126:43973 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 14:55:40.439983 12544 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:40.440066 12544 heartbeater.cc:511] Master 127.30.74.126:43973 requested a full tablet report, sending...
I20260810 14:55:40.440419 12544 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:40.440510 12544 heartbeater.cc:511] Master 127.30.74.126:43973 requested a full tablet report, sending...
W20260810 14:55:40.440883 12544 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:55:40.529950 12547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:40.530249 12547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:40.530293 12547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:40.532095 12547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:40.532171 12547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:40.532215 12547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:55:40.534658 12547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.74.84:43505/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:36413
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=42477
--tserver_master_addrs=127.30.74.126:43973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:40.534965 12547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:40.535249 12547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:40.542471 12556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:40.542798 12555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:40.543557 12547 server_base.cc:1034] running on GCE node
W20260810 14:55:40.544414 12559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:40.544685 12547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:40.544978 12547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:40.546161 12547 hybrid_clock.cc:648] HybridClock initialized: now 1786373740546117 us; error 56 us; skew 500 ppm
Aug 10 14:55:40 dist-test-slave-0b58 krb5kdc[11111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373740, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:40.549518 12547 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:55:40.550081 12547 webserver.cc:460] Webserver started at http://127.30.74.67:42477/ using document root <none> and password file <none>
I20260810 14:55:40.550374 12547 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:40.550443 12547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 14:55:40 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373740, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
I20260810 14:55:40.570034 12547 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260810 14:55:40.571293 12569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:40.571532 12547 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 14:55:40.571651 12547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "8c945aa6d7b24b0a912f6b45358b21f7"
format_stamp: "Formatted at 2026-08-10 14:55:39 on dist-test-slave-0b58"
server_key: "ba51ea4f057cfd55efa55febbcbb0380"
server_key_iv: "35ae8d9831a23f909425ad5b94b656fb"
server_key_version: "kuduclusterkey@0"
I20260810 14:55:40.572220 12547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:40.594563 12547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:40.597266 12547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:40.597498 12547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:40.597796 12547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:55:40.598242 12547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:55:40.598338 12547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:40.598410 12547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:55:40.598469 12547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:40.605257 12547 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:36413
I20260810 14:55:40.605952 12547 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:55:40.609633 12694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:36413 every 8 connection(s)
I20260810 14:55:40.611054 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 12547
I20260810 14:55:40.621333 12701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:40.611921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41194 (local address 127.30.74.126:43973)
0810 14:55:40.612276 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:40.612279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:40.612287 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:55:40.612315 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:40.612316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:40.612328 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:40.613214 (+ 886us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:40.613258 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:40.614130 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:40.615263 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:40.615559 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:40.617441 (+ 1882us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:40.617516 (+ 75us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:40.617517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:40.617517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:40.617523 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:40.619516 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:40.619924 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:40.619925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:40.619925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:40.619991 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:40.620304 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:40.620305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:40.620305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:40.620435 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:40.621089 (+ 654us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:40.621134 (+ 45us) server_negotiation.cc:299] Negotiation successful
0810 14:55:40.621221 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":317,"thread_start_us":129,"threads_started":1}
Aug 10 14:55:40 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373740, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:40.627208 12403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:40.611916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:54447 (local address 127.30.74.126:43973)
0810 14:55:40.611973 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:40.611975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:40.616655 (+ 4680us) server_negotiation.cc:407] Connection header received
0810 14:55:40.617438 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:40.617439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:40.617452 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:40.617485 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:40.619074 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:40.619838 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:40.621496 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:40.621773 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:40.621818 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:40.624147 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:40.624149 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:40.624149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:40.624157 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:40.625677 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:40.626180 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:40.626181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:40.626182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:40.626243 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:40.626515 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:40.626516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:40.626517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:40.626640 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:40.626712 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:40.627017 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 14:55:40.627118 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 14:55:40.627512 12698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:40.611577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43973 (local address 127.30.74.67:54447)
0810 14:55:40.616376 (+ 4799us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:40.616563 (+ 187us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:40.617311 (+ 748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:40.617687 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:40.617692 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:40.617916 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:40.618200 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:40.618206 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:40.619958 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:40.619959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:40.621384 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:40.621388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:40.621931 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:40.621933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:40.622118 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:40.622719 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:40.622728 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:40.624014 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:40.625774 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:40.625780 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:40.625783 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:40.626056 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:40.626317 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:40.626317 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:40.626318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:40.626436 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:40.626717 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:40.626723 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:40.626873 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 14:55:40.626983 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 14:55:40.627145 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4735,"thread_start_us":94,"threads_started":1}
I20260810 14:55:40.628015 12695 heartbeater.cc:347] Connected to a master server at 127.30.74.126:43973
I20260810 14:55:40.628156 12695 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:40.628396 12695 heartbeater.cc:511] Master 127.30.74.126:43973 requested a full tablet report, sending...
W20260810 14:55:40.629086 12695 heartbeater.cc:644] Failed to heartbeat to 127.30.74.126:43973 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 14:55:40.629177 12695 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:40.629241 12695 heartbeater.cc:511] Master 127.30.74.126:43973 requested a full tablet report, sending...
I20260810 14:55:40.629621 12695 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:40.629704 12695 heartbeater.cc:511] Master 127.30.74.126:43973 requested a full tablet report, sending...
W20260810 14:55:40.630170 12695 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 14:55:41.191361 12400 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:41.191550 12400 heartbeater.cc:511] Master 127.30.74.126:43973 requested a full tablet report, sending...
I20260810 14:55:41.192023 12184 ts_manager.cc:194] Registered new tserver with Master: 5b0e4d8276b24b33ac8c9d150c97054a (127.30.74.65:44301)
I20260810 14:55:41.192797 12184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:55975
Aug 10 14:55:41 dist-test-slave-0b58 krb5kdc[11111](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786373741, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:41.441113 12544 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:41.441306 12544 heartbeater.cc:511] Master 127.30.74.126:43973 requested a full tablet report, sending...
I20260810 14:55:41.441723 12184 ts_manager.cc:194] Registered new tserver with Master: 0889ab1831c84c3d91a71dbd12e4cac5 (127.30.74.66:33215)
I20260810 14:55:41.442289 12184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:38767
Aug 10 14:55:41 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373741, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
I20260810 14:55:41.630374 12695 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:41.630542 12695 heartbeater.cc:511] Master 127.30.74.126:43973 requested a full tablet report, sending...
I20260810 14:55:41.630893 12184 ts_manager.cc:194] Registered new tserver with Master: 8c945aa6d7b24b0a912f6b45358b21f7 (127.30.74.67:36413)
I20260810 14:55:41.631438 12184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:54447
I20260810 14:55:41.635958 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:55:41.648346 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:41.636979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41196 (local address 127.30.74.126:43973)
0810 14:55:41.637238 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:41.637241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:41.637402 (+ 161us) server_negotiation.cc:407] Connection header received
0810 14:55:41.637427 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:41.637428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:41.637439 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:41.637496 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:41.638255 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.638776 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.639837 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.640043 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.640325 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:41.644894 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:41.644896 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:41.644896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:41.644905 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:41.646768 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:41.647172 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:41.647173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:41.647173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:41.647231 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:41.647432 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:41.647433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:41.647433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:41.647567 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:41.647655 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:41.648116 (+ 461us) server_negotiation.cc:299] Negotiation successful
0810 14:55:41.648202 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":69,"threads_started":1}
I20260810 14:55:41.650553 12184 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:55:41.652858 12184 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:55:41.666255 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:41.659321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:36413 (local address 127.0.0.1:46444)
0810 14:55:41.660975 (+ 1654us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:41.660983 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:41.661020 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:41.661318 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:41.661321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:41.661324 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:55:41.661396 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:41.661398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.663034 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.663035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:41.664422 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:41.664425 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.665888 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.665889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:41.665988 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:41.666003 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 14:55:41.666074 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 14:55:41.666154 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1640,"thread_start_us":88,"threads_started":1}
I20260810 14:55:41.666610 12731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:41.659558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46444 (local address 127.30.74.67:36413)
0810 14:55:41.660681 (+ 1123us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:41.660685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:41.661073 (+ 388us) server_negotiation.cc:407] Connection header received
0810 14:55:41.661112 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:41.661115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:41.661133 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:41.661199 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:55:41.661471 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.662943 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.664885 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.665706 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.665781 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:41.665846 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:41.666485 (+ 639us) server_negotiation.cc:299] Negotiation successful
0810 14:55:41.666532 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1090,"thread_start_us":53,"threads_started":1}
I20260810 14:55:41.667338 12608 tablet_service.cc:1467] Processing CreateTablet for tablet 98ef2f28b56c4204aa170447c09881d9 (DEFAULT_TABLE table=test-table [id=1ded53c0579d4e34945124db5a3a71da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:55:41.667734 12608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98ef2f28b56c4204aa170447c09881d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:55:41.672734 12734 tablet_bootstrap.cc:492] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7: Bootstrap starting.
I20260810 14:55:41.673024 12728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:41.659947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:33215 (local address 127.0.0.1:45840)
0810 14:55:41.668350 (+ 8403us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:41.668365 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:41.668436 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:41.668746 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:41.668747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:41.668749 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:55:41.668808 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:41.668810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.670089 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.670091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:41.671181 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:41.671184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.672764 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.672765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:41.672841 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:41.672847 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:55:41.672898 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 14:55:41.672947 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8379,"thread_start_us":66,"threads_started":1}
I20260810 14:55:41.673344 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:41.660183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45840 (local address 127.30.74.66:33215)
0810 14:55:41.660597 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:41.660603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:41.668510 (+ 7907us) server_negotiation.cc:407] Connection header received
0810 14:55:41.668558 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:41.668561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:41.668580 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:41.668654 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:55:41.668896 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.669975 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.671286 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.672085 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.672172 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:41.672233 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:41.673214 (+ 981us) server_negotiation.cc:299] Negotiation successful
0810 14:55:41.673266 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":54,"threads_started":1}
I20260810 14:55:41.673882 12734 tablet_bootstrap.cc:654] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7: Neither blocks nor log segments found. Creating new log.
I20260810 14:55:41.674048 12473 tablet_service.cc:1467] Processing CreateTablet for tablet 98ef2f28b56c4204aa170447c09881d9 (DEFAULT_TABLE table=test-table [id=1ded53c0579d4e34945124db5a3a71da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:55:41.674237 12734 log.cc:824] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:41.674423 12473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98ef2f28b56c4204aa170447c09881d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:55:41.675156 12734 tablet_bootstrap.cc:492] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7: No bootstrap required, opened a new log
I20260810 14:55:41.675238 12734 ts_tablet_manager.cc:1397] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 14:55:41.678098 12734 raft_consensus.cc:348] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b0e4d8276b24b33ac8c9d150c97054a" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 44301 } } peers { permanent_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36413 } } peers { permanent_uuid: "0889ab1831c84c3d91a71dbd12e4cac5" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33215 } }
I20260810 14:55:41.678295 12734 raft_consensus.cc:374] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:55:41.678345 12734 raft_consensus.cc:729] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c945aa6d7b24b0a912f6b45358b21f7, State: Initialized, Role: FOLLOWER
I20260810 14:55:41.678458 12734 consensus_queue.cc:260] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b0e4d8276b24b33ac8c9d150c97054a" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 44301 } } peers { permanent_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36413 } } peers { permanent_uuid: "0889ab1831c84c3d91a71dbd12e4cac5" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33215 } }
I20260810 14:55:41.678731 12734 ts_tablet_manager.cc:1428] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 14:55:41.679244 12695 heartbeater.cc:503] Master 127.30.74.126:43973 was elected leader, sending a full tablet report...
I20260810 14:55:41.680658 12736 tablet_bootstrap.cc:492] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5: Bootstrap starting.
I20260810 14:55:41.681341 12736 tablet_bootstrap.cc:654] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5: Neither blocks nor log segments found. Creating new log.
I20260810 14:55:41.681658 12736 log.cc:824] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:41.682183 12732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:41.659048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:44301 (local address 127.0.0.1:58810)
0810 14:55:41.660753 (+ 1705us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:41.660804 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:41.660874 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:41.672614 (+ 11740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:41.672616 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:41.672618 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:55:41.672684 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:41.672686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.679654 (+ 6968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.679656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:41.680790 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:41.680794 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.681917 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.681918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:41.681994 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:41.681999 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:55:41.682052 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 14:55:41.682101 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1672,"thread_start_us":82,"threads_started":1}
I20260810 14:55:41.682525 12729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:41.659144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.30.74.65:44301)
0810 14:55:41.672360 (+ 13216us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:41.672366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:41.672377 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:55:41.672415 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:41.672417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:41.672436 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:41.672514 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:55:41.678425 (+ 5911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.679522 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.680893 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.681707 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.681803 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:41.681875 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:41.682399 (+ 524us) server_negotiation.cc:299] Negotiation successful
0810 14:55:41.682446 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13168,"thread_start_us":41,"threads_started":1}
I20260810 14:55:41.683171 12328 tablet_service.cc:1467] Processing CreateTablet for tablet 98ef2f28b56c4204aa170447c09881d9 (DEFAULT_TABLE table=test-table [id=1ded53c0579d4e34945124db5a3a71da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:55:41.683521 12328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98ef2f28b56c4204aa170447c09881d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:55:41.688975 12736 tablet_bootstrap.cc:492] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5: No bootstrap required, opened a new log
I20260810 14:55:41.689071 12736 ts_tablet_manager.cc:1397] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260810 14:55:41.691431 12736 raft_consensus.cc:348] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b0e4d8276b24b33ac8c9d150c97054a" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 44301 } } peers { permanent_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36413 } } peers { permanent_uuid: "0889ab1831c84c3d91a71dbd12e4cac5" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33215 } }
I20260810 14:55:41.691632 12736 raft_consensus.cc:374] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:55:41.691664 12736 raft_consensus.cc:729] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0889ab1831c84c3d91a71dbd12e4cac5, State: Initialized, Role: FOLLOWER
I20260810 14:55:41.691821 12736 consensus_queue.cc:260] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b0e4d8276b24b33ac8c9d150c97054a" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 44301 } } peers { permanent_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36413 } } peers { permanent_uuid: "0889ab1831c84c3d91a71dbd12e4cac5" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33215 } }
I20260810 14:55:41.692107 12736 ts_tablet_manager.cc:1428] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:55:41.692232 12544 heartbeater.cc:503] Master 127.30.74.126:43973 was elected leader, sending a full tablet report...
I20260810 14:55:41.696488 12739 tablet_bootstrap.cc:492] T 98ef2f28b56c4204aa170447c09881d9 P 5b0e4d8276b24b33ac8c9d150c97054a: Bootstrap starting.
I20260810 14:55:41.697269 12739 tablet_bootstrap.cc:654] T 98ef2f28b56c4204aa170447c09881d9 P 5b0e4d8276b24b33ac8c9d150c97054a: Neither blocks nor log segments found. Creating new log.
I20260810 14:55:41.697604 12739 log.cc:824] T 98ef2f28b56c4204aa170447c09881d9 P 5b0e4d8276b24b33ac8c9d150c97054a: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:41.699337 12739 tablet_bootstrap.cc:492] T 98ef2f28b56c4204aa170447c09881d9 P 5b0e4d8276b24b33ac8c9d150c97054a: No bootstrap required, opened a new log
I20260810 14:55:41.699420 12739 ts_tablet_manager.cc:1397] T 98ef2f28b56c4204aa170447c09881d9 P 5b0e4d8276b24b33ac8c9d150c97054a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 14:55:41.701726 12739 raft_consensus.cc:348] T 98ef2f28b56c4204aa170447c09881d9 P 5b0e4d8276b24b33ac8c9d150c97054a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b0e4d8276b24b33ac8c9d150c97054a" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 44301 } } peers { permanent_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36413 } } peers { permanent_uuid: "0889ab1831c84c3d91a71dbd12e4cac5" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33215 } }
I20260810 14:55:41.701929 12739 raft_consensus.cc:374] T 98ef2f28b56c4204aa170447c09881d9 P 5b0e4d8276b24b33ac8c9d150c97054a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:55:41.701968 12739 raft_consensus.cc:729] T 98ef2f28b56c4204aa170447c09881d9 P 5b0e4d8276b24b33ac8c9d150c97054a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b0e4d8276b24b33ac8c9d150c97054a, State: Initialized, Role: FOLLOWER
I20260810 14:55:41.702114 12739 consensus_queue.cc:260] T 98ef2f28b56c4204aa170447c09881d9 P 5b0e4d8276b24b33ac8c9d150c97054a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b0e4d8276b24b33ac8c9d150c97054a" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 44301 } } peers { permanent_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36413 } } peers { permanent_uuid: "0889ab1831c84c3d91a71dbd12e4cac5" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33215 } }
I20260810 14:55:41.702414 12739 ts_tablet_manager.cc:1428] T 98ef2f28b56c4204aa170447c09881d9 P 5b0e4d8276b24b33ac8c9d150c97054a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 14:55:41.702553 12400 heartbeater.cc:503] Master 127.30.74.126:43973 was elected leader, sending a full tablet report...
W20260810 14:55:41.861028 12696 tablet.cc:2369] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:55:41.899551 12737 raft_consensus.cc:482] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:55:41.899706 12737 raft_consensus.cc:504] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b0e4d8276b24b33ac8c9d150c97054a" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 44301 } } peers { permanent_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36413 } } peers { permanent_uuid: "0889ab1831c84c3d91a71dbd12e4cac5" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33215 } }
I20260810 14:55:41.900105 12737 leader_election.cc:288] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b0e4d8276b24b33ac8c9d150c97054a (127.30.74.65:44301), 0889ab1831c84c3d91a71dbd12e4cac5 (127.30.74.66:33215)
W20260810 14:55:41.909994 12546 tablet.cc:2369] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:55:41.910187 12740 raft_consensus.cc:482] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:55:41.910276 12740 raft_consensus.cc:504] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b0e4d8276b24b33ac8c9d150c97054a" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 44301 } } peers { permanent_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36413 } } peers { permanent_uuid: "0889ab1831c84c3d91a71dbd12e4cac5" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33215 } }
I20260810 14:55:41.910662 12740 leader_election.cc:288] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b0e4d8276b24b33ac8c9d150c97054a (127.30.74.65:44301), 8c945aa6d7b24b0a912f6b45358b21f7 (127.30.74.67:36413)
I20260810 14:55:41.911863 12729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:41.900823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:34931 (local address 127.30.74.65:44301)
0810 14:55:41.900931 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:41.900934 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:41.904352 (+ 3418us) server_negotiation.cc:407] Connection header received
0810 14:55:41.904397 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:41.904398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:41.904410 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:41.904483 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:55:41.904732 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.905434 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.909413 (+ 3979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.909956 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.910014 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:41.910025 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:41.911735 (+ 1710us) server_negotiation.cc:299] Negotiation successful
0810 14:55:41.911794 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 14:55:41.912148 12745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:41.900369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:44301 (local address 127.30.74.67:34931)
0810 14:55:41.903574 (+ 3205us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:41.903591 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:41.903717 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:41.904544 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:41.904545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:41.904547 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:55:41.904640 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:41.904642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.905587 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.905588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:41.909294 (+ 3706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:41.909298 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.911556 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.911557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:41.911647 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:41.911652 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:55:41.912001 (+ 349us) client_negotiation.cc:240] Negotiation successful
0810 14:55:41.912053 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3179,"thread_start_us":112,"threads_started":1}
I20260810 14:55:41.912509 12348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ef2f28b56c4204aa170447c09881d9" candidate_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b0e4d8276b24b33ac8c9d150c97054a" is_pre_election: true
I20260810 14:55:41.912712 12348 raft_consensus.cc:2454] T 98ef2f28b56c4204aa170447c09881d9 P 5b0e4d8276b24b33ac8c9d150c97054a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c945aa6d7b24b0a912f6b45358b21f7 in term 0.
I20260810 14:55:41.912751 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:41.900888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:50989 (local address 127.30.74.66:33215)
0810 14:55:41.900964 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:41.900965 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:41.903913 (+ 2948us) server_negotiation.cc:407] Connection header received
0810 14:55:41.904091 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:41.904093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:41.904105 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:41.904162 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:55:41.904524 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.905307 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.907163 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.907914 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.907967 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:41.907977 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:41.912610 (+ 4633us) server_negotiation.cc:299] Negotiation successful
0810 14:55:41.912682 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:55:41.913004 12570 leader_election.cc:302] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b0e4d8276b24b33ac8c9d150c97054a, 8c945aa6d7b24b0a912f6b45358b21f7; no voters:
I20260810 14:55:41.913151 12737 raft_consensus.cc:2781] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:55:41.913241 12737 raft_consensus.cc:482] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:55:41.913264 12737 raft_consensus.cc:3037] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:55:41.913443 12744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:41.900597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:33215 (local address 127.30.74.67:50989)
0810 14:55:41.903833 (+ 3236us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:41.903864 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:41.903966 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:41.904296 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:41.904297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:41.904299 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:55:41.904425 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:41.904428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.905414 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.905414 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:41.907000 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:41.907004 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.912414 (+ 5410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.912415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:41.912505 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:41.912508 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:55:41.913044 (+ 536us) client_negotiation.cc:240] Negotiation successful
0810 14:55:41.913362 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3221,"thread_start_us":71,"threads_started":1}
I20260810 14:55:41.914212 12737 raft_consensus.cc:504] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b0e4d8276b24b33ac8c9d150c97054a" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 44301 } } peers { permanent_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36413 } } peers { permanent_uuid: "0889ab1831c84c3d91a71dbd12e4cac5" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33215 } }
I20260810 14:55:41.918286 12737 leader_election.cc:288] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [CANDIDATE]: Term 1 election: Requested vote from peers 5b0e4d8276b24b33ac8c9d150c97054a (127.30.74.65:44301), 0889ab1831c84c3d91a71dbd12e4cac5 (127.30.74.66:33215)
I20260810 14:55:41.918579 12348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ef2f28b56c4204aa170447c09881d9" candidate_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b0e4d8276b24b33ac8c9d150c97054a"
I20260810 14:55:41.918700 12348 raft_consensus.cc:3037] T 98ef2f28b56c4204aa170447c09881d9 P 5b0e4d8276b24b33ac8c9d150c97054a [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:55:41.918756 12496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ef2f28b56c4204aa170447c09881d9" candidate_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0889ab1831c84c3d91a71dbd12e4cac5" is_pre_election: true
I20260810 14:55:41.918883 12496 raft_consensus.cc:2454] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c945aa6d7b24b0a912f6b45358b21f7 in term 0.
I20260810 14:55:41.918999 12495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ef2f28b56c4204aa170447c09881d9" candidate_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0889ab1831c84c3d91a71dbd12e4cac5"
I20260810 14:55:41.919076 12495 raft_consensus.cc:3037] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:55:41.919785 12348 raft_consensus.cc:2454] T 98ef2f28b56c4204aa170447c09881d9 P 5b0e4d8276b24b33ac8c9d150c97054a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c945aa6d7b24b0a912f6b45358b21f7 in term 1.
I20260810 14:55:41.920001 12570 leader_election.cc:302] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b0e4d8276b24b33ac8c9d150c97054a, 8c945aa6d7b24b0a912f6b45358b21f7; no voters:
I20260810 14:55:41.920003 12495 raft_consensus.cc:2454] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c945aa6d7b24b0a912f6b45358b21f7 in term 1.
I20260810 14:55:41.920104 12737 raft_consensus.cc:2781] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:55:41.920295 12737 raft_consensus.cc:686] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8c945aa6d7b24b0a912f6b45358b21f7, State: Running, Role: LEADER
I20260810 14:55:41.920413 12737 consensus_queue.cc:237] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b0e4d8276b24b33ac8c9d150c97054a" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 44301 } } peers { permanent_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36413 } } peers { permanent_uuid: "0889ab1831c84c3d91a71dbd12e4cac5" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33215 } }
I20260810 14:55:41.921335 12184 catalog_manager.cc:5495] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c945aa6d7b24b0a912f6b45358b21f7 (127.30.74.67). New cstate: current_term: 1 leader_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b0e4d8276b24b33ac8c9d150c97054a" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 44301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 36413 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0889ab1831c84c3d91a71dbd12e4cac5" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33215 } health_report { overall_health: UNKNOWN } } }
I20260810 14:55:41.923660 12746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:41.911138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:36413 (local address 127.30.74.66:33491)
0810 14:55:41.918662 (+ 7524us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:41.918674 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:41.918834 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:41.920420 (+ 1586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:41.920421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:41.920424 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:55:41.920478 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:41.920480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.921359 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.921360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:41.922519 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:41.922523 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.923371 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.923373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:41.923440 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:41.923445 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:55:41.923505 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 14:55:41.923547 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7493,"thread_start_us":57,"threads_started":1}
I20260810 14:55:41.923939 12731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:41.911449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:33491 (local address 127.30.74.67:36413)
0810 14:55:41.911502 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:41.911504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:41.920119 (+ 8615us) server_negotiation.cc:407] Connection header received
0810 14:55:41.920150 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:41.920151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:41.920162 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:41.920226 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:55:41.920591 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.921247 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.922631 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.923252 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.923329 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:41.923338 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:41.923833 (+ 495us) server_negotiation.cc:299] Negotiation successful
0810 14:55:41.923876 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 14:55:41.924248 12648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ef2f28b56c4204aa170447c09881d9" candidate_uuid: "0889ab1831c84c3d91a71dbd12e4cac5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c945aa6d7b24b0a912f6b45358b21f7" is_pre_election: true
W20260810 14:55:41.924739 12401 tablet.cc:2369] T 98ef2f28b56c4204aa170447c09881d9 P 5b0e4d8276b24b33ac8c9d150c97054a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:55:41.929949 12729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:41.911335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:37243 (local address 127.30.74.65:44301)
0810 14:55:41.911952 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:41.911954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:41.916878 (+ 4924us) server_negotiation.cc:407] Connection header received
0810 14:55:41.917035 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:41.917037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:41.917046 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:41.917074 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:55:41.917283 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.917981 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.928954 (+ 10973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.929496 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.929557 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:41.929568 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:41.929822 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 14:55:41.929876 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":73,"threads_started":1}
I20260810 14:55:41.930250 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:41.910911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:44301 (local address 127.30.74.66:37243)
0810 14:55:41.916784 (+ 5873us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:41.916800 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:41.916958 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:41.917135 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:41.917136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:41.917139 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:55:41.917214 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:41.917216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.918090 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.918091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:41.928822 (+ 10731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:41.928827 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.929654 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.929654 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:41.929732 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:41.929737 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:55:41.930103 (+ 366us) client_negotiation.cc:240] Negotiation successful
0810 14:55:41.930163 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5847,"thread_start_us":114,"threads_started":1}
I20260810 14:55:41.930490 12348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98ef2f28b56c4204aa170447c09881d9" candidate_uuid: "0889ab1831c84c3d91a71dbd12e4cac5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b0e4d8276b24b33ac8c9d150c97054a" is_pre_election: true
I20260810 14:55:41.930649 12348 raft_consensus.cc:2379] T 98ef2f28b56c4204aa170447c09881d9 P 5b0e4d8276b24b33ac8c9d150c97054a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0889ab1831c84c3d91a71dbd12e4cac5 in current term 1: Already voted for candidate 8c945aa6d7b24b0a912f6b45358b21f7 in this term.
I20260810 14:55:41.930955 12424 leader_election.cc:302] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0889ab1831c84c3d91a71dbd12e4cac5; no voters: 5b0e4d8276b24b33ac8c9d150c97054a, 8c945aa6d7b24b0a912f6b45358b21f7
I20260810 14:55:41.931154 12740 raft_consensus.cc:2726] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 10 14:55:41 dist-test-slave-0b58 krb5kdc[11111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373741, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for HTTP/127.30.74.84@KRBTEST.COM
I20260810 14:55:41.952801 12731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:41.947224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46460 (local address 127.30.74.67:36413)
0810 14:55:41.947269 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:41.947270 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:41.947278 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:55:41.947319 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:41.947320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:41.947343 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:41.947405 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:55:41.947575 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.948309 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.949359 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:41.949645 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:41.949709 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:41.952357 (+ 2648us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:55:41.952528 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:55:41.952648 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:41.952680 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 14:55:41.952711 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:55:41.957436 12495 raft_consensus.cc:1261] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5 [term 1 FOLLOWER]: Refusing update from remote peer 8c945aa6d7b24b0a912f6b45358b21f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:55:41.957471 12348 raft_consensus.cc:1261] T 98ef2f28b56c4204aa170447c09881d9 P 5b0e4d8276b24b33ac8c9d150c97054a [term 1 FOLLOWER]: Refusing update from remote peer 8c945aa6d7b24b0a912f6b45358b21f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:55:41.957800 12751 consensus_queue.cc:1035] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b0e4d8276b24b33ac8c9d150c97054a" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 44301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:55:41.958940 12737 consensus_queue.cc:1035] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0889ab1831c84c3d91a71dbd12e4cac5" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 33215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:55:41.968974 12754 mvcc.cc:205] Tried to move back new op lower bound from 7316986847053959168 to 7316986846906617856. Current Snapshot: MvccSnapshot[applied={T|T < 7316986847053959168}]
I20260810 14:55:41.970165 12757 mvcc.cc:205] Tried to move back new op lower bound from 7316986847053959168 to 7316986846906617856. Current Snapshot: MvccSnapshot[applied={T|T < 7316986847053959168}]
I20260810 14:55:41.973431 12755 mvcc.cc:205] Tried to move back new op lower bound from 7316986847053959168 to 7316986846906617856. Current Snapshot: MvccSnapshot[applied={T|T < 7316986847053959168 or (T in {7316986847053959168})}]
I20260810 14:55:41.976868 12184 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:55:41.977006 12184 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:55:41.978088 12184 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63d9f079cd3949609ed70713d524c113: Sending DeleteTablet for 3 replicas of tablet 98ef2f28b56c4204aa170447c09881d9
I20260810 14:55:41.978571 12608 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ef2f28b56c4204aa170447c09881d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:55:41 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:46444
I20260810 14:55:41.978861 12328 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ef2f28b56c4204aa170447c09881d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:55:41 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:58810
I20260810 14:55:41.979456 12473 tablet_service.cc:1514] Processing DeleteTablet for tablet 98ef2f28b56c4204aa170447c09881d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:55:41 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:45840
I20260810 14:55:41.980271 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 12226
I20260810 14:55:41.985517 12765 tablet_replica.cc:330] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5: stopping tablet replica
I20260810 14:55:41.985751 12765 raft_consensus.cc:2229] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:55:41.986024 12765 raft_consensus.cc:2258] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:55:41.986267 12763 tablet_replica.cc:330] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7: stopping tablet replica
I20260810 14:55:41.987124 12763 raft_consensus.cc:2229] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:55:41.987812 12765 ts_tablet_manager.cc:1905] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:55:41.988044 12763 raft_consensus.cc:2258] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:55:41.994683 12765 ts_tablet_manager.cc:1918] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:55:41.995491 12763 ts_tablet_manager.cc:1905] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:55:41.997336 12763 ts_tablet_manager.cc:1918] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:55:41.994807 12765 log.cc:1196] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/98ef2f28b56c4204aa170447c09881d9
I20260810 14:55:41.997902 12765 ts_tablet_manager.cc:1939] T 98ef2f28b56c4204aa170447c09881d9 P 0889ab1831c84c3d91a71dbd12e4cac5: Deleting consensus metadata
I20260810 14:55:41.998031 12763 log.cc:1196] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/98ef2f28b56c4204aa170447c09881d9
I20260810 14:55:41.998454 12763 ts_tablet_manager.cc:1939] T 98ef2f28b56c4204aa170447c09881d9 P 8c945aa6d7b24b0a912f6b45358b21f7: Deleting consensus metadata
I20260810 14:55:41.999032 12170 catalog_manager.cc:4841] TS 0889ab1831c84c3d91a71dbd12e4cac5 (127.30.74.66:33215): tablet 98ef2f28b56c4204aa170447c09881d9 (table test-table [id=1ded53c0579d4e34945124db5a3a71da]) successfully deleted
I20260810 14:55:41.999600 12169 catalog_manager.cc:4841] TS 8c945aa6d7b24b0a912f6b45358b21f7 (127.30.74.67:36413): tablet 98ef2f28b56c4204aa170447c09881d9 (table test-table [id=1ded53c0579d4e34945124db5a3a71da]) successfully deleted
W20260810 14:55:42.000173 12168 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:44301 (error 108)
I20260810 14:55:42.008445 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 12404
I20260810 14:55:42.016641 12732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:42.016458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:44301 (local address 127.0.0.1:58826)
0810 14:55:42.016533 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:42.016590 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:44301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 14:55:42.016837 12168 catalog_manager.cc:4584] TS 5b0e4d8276b24b33ac8c9d150c97054a (127.30.74.65:44301): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 98ef2f28b56c4204aa170447c09881d9: Network error: Client connection negotiation failed: client connection to 127.30.74.65:44301: connect: Connection refused (error 111)
I20260810 14:55:42.018018 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 12547
I20260810 14:55:42.026372 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 12149
I20260810 14:55:42.037976 31017 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 14:55:42.383285 31017 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 14:55:42.383430 31017 mini_ranger.cc:66] Stopping Ranger...
I20260810 14:55:42.728529 31017 mini_ranger.cc:68] Stopped Ranger
2026-08-10 14:55:42.728 UTC [11144] LOG: received smart shutdown request
2026-08-10 14:55:42.734 UTC [11144] LOG: background worker "logical replication launcher" (PID 11153) exited with exit code 1
2026-08-10 14:55:42.734 UTC [11148] LOG: shutting down
2026-08-10 14:55:42.922 UTC [11144] LOG: database system is shut down
2026-08-10T14:55:42Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49369 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:55:43 dist-test-slave-0b58 krb5kdc[12776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:55:43 dist-test-slave-0b58 krb5kdc[12776](info): set up 2 sockets
Aug 10 14:55:43 dist-test-slave-0b58 krb5kdc[12776](info): commencing operation
krb5kdc: starting...
W20260810 14:55:44.162919 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:55:44 dist-test-slave-0b58 krb5kdc[12776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:55:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:55:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:55:44.296025 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:43031
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:41601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:43031,127.30.74.125:32785,127.30.74.124:34925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:55:44.381325 12789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:44.381876 12789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:44.381906 12789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:44.383240 12789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:55:44.383292 12789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:44.383308 12789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:44.383327 12789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:55:44.383340 12789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:55:44.385156 12789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:41601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:43031,127.30.74.125:32785,127.30.74.124:34925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:43031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:44.385435 12789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:44.385634 12789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:44.390825 12794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:44.390825 12795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:44.390820 12797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:44.391216 12789 server_base.cc:1034] running on GCE node
I20260810 14:55:44.391484 12789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:44.391808 12789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:44.392984 12789 hybrid_clock.cc:648] HybridClock initialized: now 1786373744392970 us; error 45 us; skew 500 ppm
Aug 10 14:55:44 dist-test-slave-0b58 krb5kdc[12776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:44.395581 12789 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:55:44.396003 12789 webserver.cc:460] Webserver started at http://127.30.74.126:44519/ using document root <none> and password file <none>
I20260810 14:55:44.396200 12789 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:44.396241 12789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:44.396418 12789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:55:44.397467 12789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "808ca1b71b46464ea4b0d1d500ac3609"
format_stamp: "Formatted at 2026-08-10 14:55:44 on dist-test-slave-0b58"
server_key: "bbaf540fd59619fd339b24a1b6a66a02"
server_key_iv: "22cc5c20fc70b027949a963782cd94f7"
server_key_version: "encryptionkey@0"
I20260810 14:55:44.397776 12789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "808ca1b71b46464ea4b0d1d500ac3609"
format_stamp: "Formatted at 2026-08-10 14:55:44 on dist-test-slave-0b58"
server_key: "bbaf540fd59619fd339b24a1b6a66a02"
server_key_iv: "22cc5c20fc70b027949a963782cd94f7"
server_key_version: "encryptionkey@0"
I20260810 14:55:44.399111 12789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:55:44.399842 12805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:44.400039 12789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:55:44.400141 12789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "808ca1b71b46464ea4b0d1d500ac3609"
format_stamp: "Formatted at 2026-08-10 14:55:44 on dist-test-slave-0b58"
server_key: "bbaf540fd59619fd339b24a1b6a66a02"
server_key_iv: "22cc5c20fc70b027949a963782cd94f7"
server_key_version: "encryptionkey@0"
I20260810 14:55:44.400206 12789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:44.434149 12789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:44.436708 12789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:44.436857 12789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:44.441440 12789 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:43031
I20260810 14:55:44.441525 12862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:43031 every 8 connection(s)
I20260810 14:55:44.441941 12789 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:55:44.442691 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 12789
I20260810 14:55:44.442826 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:55:44.443073 31017 external_mini_cluster.cc:1428] Setting key 91857e25ffbc33d719b10e8b9c8c4028
I20260810 14:55:44.443219 12863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:55:44.445698 12863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.74.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.125@KRBTEST.COM" created.
Aug 10 14:55:44 dist-test-slave-0b58 krb5kdc[12776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:44.459975 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.446497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43031 (local address 127.0.0.1:34132)
0810 14:55:44.446809 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.446840 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:44.447297 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:44.447635 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:44.447638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:44.447989 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:44.448164 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.448169 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.449920 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.449921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.450634 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.450636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.451392 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.451393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.451459 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.452040 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:44.452049 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:44.456217 (+ 4168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:44.458322 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.458330 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.458332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.458542 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.458902 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.458903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.458903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.459013 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.459502 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:44.459507 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:44.459570 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 14:55:44.459646 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 14:55:44.459794 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":112,"threads_started":1}
I20260810 14:55:44.460155 12867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.446562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34132 (local address 127.30.74.126:43031)
0810 14:55:44.446747 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:44.446759 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:44.446943 (+ 184us) server_negotiation.cc:407] Connection header received
0810 14:55:44.447469 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:44.447472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:44.447497 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:44.447627 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:44.448309 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.449794 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.450729 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.451291 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.451428 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.456371 (+ 4943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:44.456382 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:44.456383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:44.456391 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:44.458186 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.458635 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.458639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.458640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.458806 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.459083 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.459084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.459085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.459428 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:44.459519 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:44.459727 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 14:55:44.459901 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260810 14:55:44.461068 12863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:55:44.461473 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.461349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:32785 (local address 127.0.0.1:37088)
0810 14:55:44.461402 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.461453 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:32785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 14:55:44.461608 12810 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.74.125:32785: connect: Connection refused (error 111)
I20260810 14:55:44.462057 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.461958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:32785 (local address 127.0.0.1:37094)
0810 14:55:44.462003 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.462028 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:32785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 14:55:44.462153 12863 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.125:32785: Network error: Client connection negotiation failed: client connection to 127.30.74.125:32785: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.125.keytab.
Entry for principal kudu/127.30.74.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.125.keytab.
I20260810 14:55:44.473245 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.125:32785
--webserver_interface=127.30.74.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.125.keytab
--principal=kudu/127.30.74.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:41601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:43031,127.30.74.125:32785,127.30.74.124:34925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:55:44.511392 12863 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } attempt: 1
I20260810 14:55:44.511922 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.511779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:32785 (local address 127.0.0.1:37102)
0810 14:55:44.511850 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.511892 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:32785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 14:55:44.512318 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.512192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:32785 (local address 127.0.0.1:37114)
0810 14:55:44.512239 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.512291 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:32785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 14:55:44.512403 12863 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.125:32785: Network error: Client connection negotiation failed: client connection to 127.30.74.125:32785: connect: Connection refused (error 111)
W20260810 14:55:44.558151 12870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:44.558384 12870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:44.558409 12870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:44.559674 12870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:55:44.559723 12870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:44.559737 12870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:44.559751 12870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:55:44.559763 12870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:55:44.561461 12870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:41601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:43031,127.30.74.125:32785,127.30.74.124:34925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.125.keytab
--principal=kudu/127.30.74.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.125:32785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.74.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:44.561734 12870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:44.562016 12870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:44.567205 12879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:44.567315 12877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:44.567198 12876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:44.567365 12870 server_base.cc:1034] running on GCE node
I20260810 14:55:44.567680 12870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:44.568025 12870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:44.569232 12870 hybrid_clock.cc:648] HybridClock initialized: now 1786373744569202 us; error 53 us; skew 500 ppm
Aug 10 14:55:44 dist-test-slave-0b58 krb5kdc[12776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:44.572175 12870 init.cc:377] Logged in from keytab as kudu/127.30.74.125@KRBTEST.COM (short username kudu)
I20260810 14:55:44.572767 12870 webserver.cc:460] Webserver started at http://127.30.74.125:42095/ using document root <none> and password file <none>
I20260810 14:55:44.573046 12870 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:44.573098 12870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:44.573275 12870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:55:44.574448 12870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/data/instance:
uuid: "4235617109b043efb90b0c6c066cccaa"
format_stamp: "Formatted at 2026-08-10 14:55:44 on dist-test-slave-0b58"
server_key: "904dbf3a3ec842cfe7440b2fa991cd75"
server_key_iv: "2464a5fb7a1eb8b6e2322698d9bf28eb"
server_key_version: "encryptionkey@0"
I20260810 14:55:44.574864 12870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/wal/instance:
uuid: "4235617109b043efb90b0c6c066cccaa"
format_stamp: "Formatted at 2026-08-10 14:55:44 on dist-test-slave-0b58"
server_key: "904dbf3a3ec842cfe7440b2fa991cd75"
server_key_iv: "2464a5fb7a1eb8b6e2322698d9bf28eb"
server_key_version: "encryptionkey@0"
I20260810 14:55:44.576419 12870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:55:44.577333 12888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:44.577553 12870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:44.577656 12870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/wal
uuid: "4235617109b043efb90b0c6c066cccaa"
format_stamp: "Formatted at 2026-08-10 14:55:44 on dist-test-slave-0b58"
server_key: "904dbf3a3ec842cfe7440b2fa991cd75"
server_key_iv: "2464a5fb7a1eb8b6e2322698d9bf28eb"
server_key_version: "encryptionkey@0"
I20260810 14:55:44.577721 12870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:44.580538 12863 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } attempt: 2
I20260810 14:55:44.581017 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.580884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:32785 (local address 127.0.0.1:37116)
0810 14:55:44.580946 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.580987 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:32785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 14:55:44.581486 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.581367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:32785 (local address 127.0.0.1:37126)
0810 14:55:44.581430 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.581459 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:32785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 14:55:44.581590 12863 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.125:32785: Network error: Client connection negotiation failed: client connection to 127.30.74.125:32785: connect: Connection refused (error 111)
I20260810 14:55:44.606416 12870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:44.609141 12870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:44.609313 12870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:44.613543 12870 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.125:32785
I20260810 14:55:44.613580 12942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.125:32785 every 8 connection(s)
I20260810 14:55:44.614001 12870 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/data/info.pb
I20260810 14:55:44.615025 12943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:55:44.616856 12943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:55:44.619801 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 12870
I20260810 14:55:44.619961 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-1/wal/instance
I20260810 14:55:44.620152 31017 external_mini_cluster.cc:1428] Setting key ba67951014e268e5cd6e210583bbe75f
Aug 10 14:55:44 dist-test-slave-0b58 krb5kdc[12776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.125@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:44.627775 12867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.617672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.30.74.126:43031)
0810 14:55:44.617733 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:44.617735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:44.617939 (+ 204us) server_negotiation.cc:407] Connection header received
0810 14:55:44.618403 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:44.618404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:44.618420 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:44.618488 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:44.619143 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.619653 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.621229 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.621463 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.621528 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.623987 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:44.623989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:44.623989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:44.623999 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:44.626087 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.626603 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.626604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.626605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.626691 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.626972 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.626973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.626974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.627147 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:44.627223 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:44.627528 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 14:55:44.627645 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 14:55:44.627830 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.617584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43031 (local address 127.0.0.1:34148)
0810 14:55:44.617773 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.617818 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:44.618282 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:44.618568 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:44.618571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:44.618837 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:44.619044 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.619048 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.619760 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.619761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.621111 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.621115 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.621561 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.621562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.621739 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.622418 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:44.622427 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:44.623874 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:44.626228 (+ 2354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.626239 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.626241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.626512 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.626771 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.626772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.626772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.626891 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.627227 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:44.627235 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:44.627385 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 14:55:44.627481 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 14:55:44.627637 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":129,"threads_started":1}
I20260810 14:55:44.628532 12943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.74.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.124@KRBTEST.COM" created.
Aug 10 14:55:44 dist-test-slave-0b58 krb5kdc[12776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.125@KRBTEST.COM for kudu/127.30.74.125@KRBTEST.COM
I20260810 14:55:44.637535 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.628849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:32785 (local address 127.0.0.1:37138)
0810 14:55:44.628917 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.628931 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:44.628980 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:44.629252 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:44.629253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:44.629432 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:44.629508 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.629510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.630504 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.630505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.631115 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.631117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.631884 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.631885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.631949 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.632561 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:44.632566 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:44.634188 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:44.636138 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.636140 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.636140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.636443 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.636718 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.636719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.636720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.636798 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.637172 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:44.637172 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:44.637237 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 14:55:44.637358 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 14:55:44.637456 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 14:55:44.637784 12947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.628891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37138 (local address 127.30.74.125:32785)
0810 14:55:44.629085 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:44.629089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:44.629117 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:55:44.629137 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:44.629139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:44.629151 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:44.629220 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:44.629588 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.630397 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.631214 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.631761 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.631868 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.634314 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:44.634326 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:44.634326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:44.634334 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:44.636030 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.636559 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.636563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.636563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.636637 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.636891 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.636892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.636893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.637077 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:44.637175 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:44.637432 (+ 257us) server_negotiation.cc:299] Negotiation successful
0810 14:55:44.637526 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":128,"threads_started":1}
I20260810 14:55:44.638476 12943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:55:44.638896 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.638764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34925 (local address 127.0.0.1:35236)
0810 14:55:44.638817 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.638874 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 14:55:44.639101 12892 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
I20260810 14:55:44.639631 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.639532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34925 (local address 127.0.0.1:35248)
0810 14:55:44.639586 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.639612 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 14:55:44.639770 12943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.124:34925: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.124.keytab.
Entry for principal kudu/127.30.74.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.124.keytab.
I20260810 14:55:44.651612 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.124:34925
--webserver_interface=127.30.74.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.124.keytab
--principal=kudu/127.30.74.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:41601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:43031,127.30.74.125:32785,127.30.74.124:34925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:55:44.672737 12863 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } attempt: 3
Aug 10 14:55:44 dist-test-slave-0b58 krb5kdc[12776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for kudu/127.30.74.125@KRBTEST.COM
I20260810 14:55:44.680961 12947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.673279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37150 (local address 127.30.74.125:32785)
0810 14:55:44.673366 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:44.673369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:44.673387 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:55:44.673470 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:44.673471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:44.673486 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:44.673552 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:44.673889 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.674675 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.675425 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.675749 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.675833 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.678028 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:44.678030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:44.678030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:44.678038 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:44.679584 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.680010 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.680011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.680011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.680068 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.680355 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.680379 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.680379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.680528 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:44.680594 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:44.680770 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 14:55:44.680878 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 14:55:44.680981 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.673206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:32785 (local address 127.0.0.1:37150)
0810 14:55:44.673322 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.673339 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:44.673400 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:44.673566 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:44.673567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:44.673730 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:44.673805 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.673806 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.674797 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.674798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.675325 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.675326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.675831 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.675833 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.675899 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.676519 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:44.676525 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:44.677921 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:44.679685 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.679686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.679687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.679931 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.680135 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.680136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.680136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.680206 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.680599 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:44.680600 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:44.680661 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 14:55:44.680739 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 14:55:44.680878 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 14:55:44.681547 12863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:55:44.681979 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.681856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34925 (local address 127.0.0.1:35250)
0810 14:55:44.681911 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.681951 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 14:55:44.682400 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.682304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34925 (local address 127.0.0.1:35258)
0810 14:55:44.682352 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.682381 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 14:55:44.682544 12863 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.124:34925: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
I20260810 14:55:44.689033 12943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } attempt: 1
I20260810 14:55:44.689558 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.689413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34925 (local address 127.0.0.1:35272)
0810 14:55:44.689480 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.689524 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 14:55:44.689905 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.689800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34925 (local address 127.0.0.1:35280)
0810 14:55:44.689849 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.689876 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 14:55:44.689998 12943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.124:34925: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
I20260810 14:55:44.713712 12863 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } attempt: 1
I20260810 14:55:44.714321 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.714147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34925 (local address 127.0.0.1:35288)
0810 14:55:44.714228 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.714282 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 14:55:44.714792 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.714679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34925 (local address 127.0.0.1:35298)
0810 14:55:44.714735 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.714767 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 14:55:44.714937 12863 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.124:34925: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
W20260810 14:55:44.746651 12950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:44.746920 12950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:44.746960 12950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:44.748723 12950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:55:44.748796 12950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:44.748819 12950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:44.748853 12950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:55:44.748881 12950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:55:44.750577 12950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:41601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:43031,127.30.74.125:32785,127.30.74.124:34925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.124.keytab
--principal=kudu/127.30.74.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.124:34925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.74.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:44.750906 12950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:44.751171 12950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:44.756206 12955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:44.756203 12958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:44.756436 12950 server_base.cc:1034] running on GCE node
W20260810 14:55:44.756229 12956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:44.756830 12950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:44.757143 12950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:44.758178 12943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } attempt: 2
I20260810 14:55:44.758314 12950 hybrid_clock.cc:648] HybridClock initialized: now 1786373744758276 us; error 45 us; skew 500 ppm
I20260810 14:55:44.758613 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.758473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34925 (local address 127.0.0.1:35314)
0810 14:55:44.758540 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.758578 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 14:55:44.758944 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.758843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34925 (local address 127.0.0.1:35324)
0810 14:55:44.758890 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.758915 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 14:55:44.759033 12943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.124:34925: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
Aug 10 14:55:44 dist-test-slave-0b58 krb5kdc[12776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:44.761137 12950 init.cc:377] Logged in from keytab as kudu/127.30.74.124@KRBTEST.COM (short username kudu)
I20260810 14:55:44.761590 12950 webserver.cc:460] Webserver started at http://127.30.74.124:44975/ using document root <none> and password file <none>
I20260810 14:55:44.761786 12950 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:44.761830 12950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:44.761926 12950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:55:44.762995 12950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/data/instance:
uuid: "a070e409243649c9a94fe74acf073d1a"
format_stamp: "Formatted at 2026-08-10 14:55:44 on dist-test-slave-0b58"
server_key: "828ad86d2cbac8970a40cc0bb9baae90"
server_key_iv: "5f44050256a28698be55568458391797"
server_key_version: "encryptionkey@0"
I20260810 14:55:44.763361 12950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/wal/instance:
uuid: "a070e409243649c9a94fe74acf073d1a"
format_stamp: "Formatted at 2026-08-10 14:55:44 on dist-test-slave-0b58"
server_key: "828ad86d2cbac8970a40cc0bb9baae90"
server_key_iv: "5f44050256a28698be55568458391797"
server_key_version: "encryptionkey@0"
I20260810 14:55:44.764917 12950 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:55:44.765748 12965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:44.766039 12950 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:55:44.766160 12950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/wal
uuid: "a070e409243649c9a94fe74acf073d1a"
format_stamp: "Formatted at 2026-08-10 14:55:44 on dist-test-slave-0b58"
server_key: "828ad86d2cbac8970a40cc0bb9baae90"
server_key_iv: "5f44050256a28698be55568458391797"
server_key_version: "encryptionkey@0"
I20260810 14:55:44.766237 12950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:44.790172 12863 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } attempt: 2
I20260810 14:55:44.790694 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.790552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34925 (local address 127.0.0.1:35328)
0810 14:55:44.790622 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.790661 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 14:55:44.791076 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.790980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34925 (local address 127.0.0.1:35334)
0810 14:55:44.791028 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.791059 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 14:55:44.791167 12863 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.124:34925: Network error: Client connection negotiation failed: client connection to 127.30.74.124:34925: connect: Connection refused (error 111)
I20260810 14:55:44.791162 12950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:44.793385 12950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:44.793567 12950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:44.797744 12950 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.124:34925
I20260810 14:55:44.797839 13020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.124:34925 every 8 connection(s)
I20260810 14:55:44.798218 12950 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/data/info.pb
I20260810 14:55:44.798399 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 12950
I20260810 14:55:44.798554 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373586526129-31017-0/minicluster-data/master-2/wal/instance
I20260810 14:55:44.798851 31017 external_mini_cluster.cc:1428] Setting key a8a0f2470690e2bd206ae621939084ba
I20260810 14:55:44.799377 13021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:55:44.801942 13021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:55:44 dist-test-slave-0b58 krb5kdc[12776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.124@KRBTEST.COM
Aug 10 14:55:44 dist-test-slave-0b58 krb5kdc[12776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.124@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:44.810570 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.799399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35342 (local address 127.30.74.124:34925)
0810 14:55:44.799592 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:44.799599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:44.799625 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:55:44.800069 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:44.800077 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:44.800125 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:44.800328 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:44.801314 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.802630 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.803476 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.804062 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.804143 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.806915 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:44.806922 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:44.806923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:44.806931 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:44.808737 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.809161 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.809164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.809164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.809255 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.809603 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.809604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.809604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.809857 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:44.809967 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:44.810163 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 14:55:44.810329 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260810 14:55:44.811230 13025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.802638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43031 (local address 127.0.0.1:34152)
0810 14:55:44.802851 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.802893 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:44.802976 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:44.803133 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:44.803136 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:44.803407 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:44.803511 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.803513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.804356 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.804357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.805094 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.805097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.805612 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.805613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.805672 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.806171 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:44.806181 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:44.807994 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:44.809786 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.809793 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.809796 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.810072 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.810304 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.810305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.810305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.810379 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.810906 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:44.810909 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:44.810951 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 14:55:44.811022 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 14:55:44.811106 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":134,"threads_started":1}
I20260810 14:55:44.811489 12867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.802707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34152 (local address 127.30.74.126:43031)
0810 14:55:44.802764 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:44.802766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:44.802960 (+ 194us) server_negotiation.cc:407] Connection header received
0810 14:55:44.803040 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:44.803042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:44.803053 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:44.803112 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:44.803601 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.804240 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.805194 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.805521 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.805601 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.808106 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:44.808108 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:44.808108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:44.808116 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:44.809708 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.810154 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.810155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.810155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.810220 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.810451 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.810452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.810452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.810591 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:44.810665 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:44.811115 (+ 450us) server_negotiation.cc:299] Negotiation successful
0810 14:55:44.811219 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 14:55:44.812057 13021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:55:44 dist-test-slave-0b58 krb5kdc[12776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.124@KRBTEST.COM for kudu/127.30.74.125@KRBTEST.COM
Aug 10 14:55:44 dist-test-slave-0b58 krb5kdc[12776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:44.819121 12947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.812417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37160 (local address 127.30.74.125:32785)
0810 14:55:44.812463 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:44.812465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:44.812474 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:55:44.812500 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:44.812501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:44.812511 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:44.812566 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:44.812849 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.813392 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.814093 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.814311 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.814379 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.816499 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:44.816500 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:44.816501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:44.816507 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:44.818005 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.818385 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.818386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.818387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.818457 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.818610 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.818610 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.818611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.818773 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:44.818848 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:44.818959 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 14:55:44.819050 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 14:55:44.819169 13025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.812344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:32785 (local address 127.0.0.1:37160)
0810 14:55:44.812377 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.812386 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:44.812428 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:44.812584 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:44.812585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:44.812734 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:44.812771 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.812773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.813476 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.813477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.814009 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.814011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.814385 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.814386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.814430 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.815025 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:44.815030 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:44.816401 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:44.818090 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.818091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.818092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.818305 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.818526 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.818526 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.818526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.818572 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.818842 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:44.818843 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:44.818881 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 14:55:44.818933 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 14:55:44.819057 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260810 14:55:44.819658 13021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:55:44.820302 12867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.811689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34168 (local address 127.30.74.126:43031)
0810 14:55:44.811727 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:44.811729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:44.811738 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:55:44.811804 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:44.811805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:44.811815 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:44.811859 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:44.812536 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.813148 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.813920 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.814206 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.814259 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.817190 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:44.817191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:44.817192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:44.817199 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:44.818929 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.819323 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.819324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.819324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.819387 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.819592 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.819593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.819593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.819739 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:44.819960 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:44.820066 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 14:55:44.820157 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 10 14:55:44 dist-test-slave-0b58 krb5kdc[12776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.124@KRBTEST.COM for kudu/127.30.74.124@KRBTEST.COM
Aug 10 14:55:44 dist-test-slave-0b58 krb5kdc[12776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.125@KRBTEST.COM
I20260810 14:55:44.826900 13025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.819943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34925 (local address 127.0.0.1:35352)
0810 14:55:44.819995 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.820005 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:44.820043 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:44.820195 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:44.820196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:44.820376 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:44.820416 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.820418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.821176 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.821177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.821914 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.821917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.822396 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.822397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.822446 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.822834 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:44.822839 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:44.824178 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:44.825736 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.825737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.825737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.825952 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.826168 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.826169 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.826170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.826243 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.826554 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:44.826555 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:44.826601 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 14:55:44.826676 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 14:55:44.826786 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 14:55:44.826996 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.819993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35352 (local address 127.30.74.124:34925)
0810 14:55:44.820037 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:44.820039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:44.820048 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:55:44.820105 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:44.820105 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:44.820117 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:44.820168 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:44.820500 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.821092 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.821993 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.822301 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.822382 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.824289 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:44.824291 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:44.824292 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:44.824299 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:44.825637 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.826037 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.826038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.826038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.826088 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.826318 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.826319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.826320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.826457 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:44.826535 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:44.826718 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 14:55:44.826851 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":22}
I20260810 14:55:44.829020 13021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a: Bootstrap starting.
I20260810 14:55:44.829348 12947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.821020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37174 (local address 127.30.74.125:32785)
0810 14:55:44.821063 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:44.821065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:44.821073 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:55:44.821096 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:44.821097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:44.821107 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:44.821172 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:44.821721 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.822347 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.823257 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.823435 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.823542 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.826303 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:44.826305 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:44.826305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:44.826314 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:44.828103 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.828531 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.828532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.828532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.828594 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.828790 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.828790 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.828791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.828934 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:44.829134 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:44.829185 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 14:55:44.829266 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 14:55:44.830158 13021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a: Neither blocks nor log segments found. Creating new log.
I20260810 14:55:44.830543 13021 log.cc:824] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:44.831403 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:55:44.831466 13021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a: No bootstrap required, opened a new log
I20260810 14:55:44.833334 13021 raft_consensus.cc:348] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } }
I20260810 14:55:44.833644 13021 raft_consensus.cc:374] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:55:44.833758 13021 raft_consensus.cc:729] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a070e409243649c9a94fe74acf073d1a, State: Initialized, Role: FOLLOWER
I20260810 14:55:44.833993 13021 consensus_queue.cc:260] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } }
I20260810 14:55:44.834431 13021 sys_catalog.cc:564] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a [sys.catalog]: configured and running, proceeding with master startup.
W20260810 14:55:44.836516 13047 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:55:44.836623 13047 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:55:44.841116 12867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.832874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34170 (local address 127.30.74.126:43031)
0810 14:55:44.832922 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:44.832923 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:44.833041 (+ 118us) server_negotiation.cc:407] Connection header received
0810 14:55:44.833172 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:44.833174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:44.833188 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:44.833249 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:44.834002 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.834701 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.835858 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.836139 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.836185 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.837923 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:44.837924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:44.837924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:44.837933 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:44.839649 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.840080 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.840081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.840081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.840145 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.840433 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.840434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.840435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.840578 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:44.840653 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:44.840893 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 14:55:44.840989 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 14:55:44.841562 12947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.833065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37176 (local address 127.30.74.125:32785)
0810 14:55:44.833104 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:44.833106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:44.833407 (+ 301us) server_negotiation.cc:407] Connection header received
0810 14:55:44.833523 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:44.833525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:44.833535 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:44.833583 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:44.834333 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.835013 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.835930 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.836216 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.836313 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.838169 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:44.838171 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:44.838171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:44.838178 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:44.840139 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.840626 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.840627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.840628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.840688 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.840916 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.840917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.840917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.841053 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:44.841158 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:44.841393 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 14:55:44.841479 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 14:55:44.842173 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.833071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35364 (local address 127.30.74.124:34925)
0810 14:55:44.833110 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:44.833112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:44.833266 (+ 154us) server_negotiation.cc:407] Connection header received
0810 14:55:44.833322 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:44.833323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:44.833335 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:44.834075 (+ 740us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:44.834110 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.834844 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.835871 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.836153 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.836208 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.838653 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:44.838654 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:44.838655 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:44.838662 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:44.840776 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.841143 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.841143 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.841144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.841199 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.841407 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.841407 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.841408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.841564 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:44.841659 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:44.842000 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 14:55:44.842086 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 14:55:44.838882 12950 master_runner.cc:186] Error getting master registration for 127.30.74.126:43031: OK, instance_id { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" instance_seqno: 1786373744439968 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:55:44.844300 12950 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 14:55:44.844421 13034 sys_catalog.cc:455] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } } }
I20260810 14:55:44.845238 13034 sys_catalog.cc:458] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:55:44.850219 12943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } attempt: 3
Aug 10 14:55:44 dist-test-slave-0b58 krb5kdc[12776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.125@KRBTEST.COM for kudu/127.30.74.124@KRBTEST.COM
I20260810 14:55:44.858122 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.850566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34925 (local address 127.0.0.1:35366)
0810 14:55:44.850632 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.850645 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:44.850719 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:44.850906 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:44.850907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:44.851110 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:44.851157 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.851159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.851831 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.851832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.852649 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.852652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.853094 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.853095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.853153 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.853690 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:44.853696 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:44.855231 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:44.856986 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.856987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.856988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.857233 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.857449 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.857450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.857451 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.857522 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.857785 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:44.857786 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:44.857838 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 14:55:44.857911 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 14:55:44.858016 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 14:55:44.858122 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.850682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35366 (local address 127.30.74.124:34925)
0810 14:55:44.850756 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:44.850758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:44.850786 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:55:44.850816 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:44.850816 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:44.850826 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:44.850887 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:44.851240 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.851732 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.852740 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.853029 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.853098 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.855337 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:44.855339 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:44.855340 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:44.855349 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:44.856892 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.857317 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.857318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.857319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.857377 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.857593 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.857594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.857594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.857717 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:44.857774 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:44.857938 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 14:55:44.858012 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 14:55:44.860154 12943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa: Bootstrap starting.
I20260810 14:55:44.860905 12943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa: Neither blocks nor log segments found. Creating new log.
I20260810 14:55:44.861227 12943 log.cc:824] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:44.862074 12943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa: No bootstrap required, opened a new log
I20260810 14:55:44.863535 12943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } }
I20260810 14:55:44.863684 12943 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:55:44.863752 12943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4235617109b043efb90b0c6c066cccaa, State: Initialized, Role: FOLLOWER
I20260810 14:55:44.863910 12943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } }
I20260810 14:55:44.864212 13051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } } }
I20260810 14:55:44.864320 13051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:55:44.864326 12943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [sys.catalog]: configured and running, proceeding with master startup.
W20260810 14:55:44.866274 13063 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:55:44.866400 13063 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:55:44.866518 12870 master_runner.cc:186] Error getting master registration for 127.30.74.126:43031: OK, instance_id { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" instance_seqno: 1786373744439968 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:55:44.869086 12870 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 14:55:44.882943 13051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:55:44.883101 13051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } }
I20260810 14:55:44.883432 13051 leader_election.cc:288] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 808ca1b71b46464ea4b0d1d500ac3609 (127.30.74.126:43031), a070e409243649c9a94fe74acf073d1a (127.30.74.124:34925)
I20260810 14:55:44.883922 12995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4235617109b043efb90b0c6c066cccaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a070e409243649c9a94fe74acf073d1a" is_pre_election: true
I20260810 14:55:44.884102 12995 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4235617109b043efb90b0c6c066cccaa in term 0.
I20260810 14:55:44.884364 12892 leader_election.cc:302] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4235617109b043efb90b0c6c066cccaa, a070e409243649c9a94fe74acf073d1a; no voters:
I20260810 14:55:44.884585 13051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:55:44.884639 13051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:55:44.884662 13051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:55:44.885665 13051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } }
I20260810 14:55:44.885805 13051 leader_election.cc:288] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [CANDIDATE]: Term 1 election: Requested vote from peers 808ca1b71b46464ea4b0d1d500ac3609 (127.30.74.126:43031), a070e409243649c9a94fe74acf073d1a (127.30.74.124:34925)
I20260810 14:55:44.886044 12995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4235617109b043efb90b0c6c066cccaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a070e409243649c9a94fe74acf073d1a"
I20260810 14:55:44.886152 12995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:55:44.887104 12995 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4235617109b043efb90b0c6c066cccaa in term 1.
I20260810 14:55:44.887327 12892 leader_election.cc:302] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4235617109b043efb90b0c6c066cccaa, a070e409243649c9a94fe74acf073d1a; no voters:
I20260810 14:55:44.887909 13051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:55:44.888113 13051 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [term 1 LEADER]: Becoming Leader. State: Replica: 4235617109b043efb90b0c6c066cccaa, State: Running, Role: LEADER
I20260810 14:55:44.888211 13051 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } }
I20260810 14:55:44.888674 13065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4235617109b043efb90b0c6c066cccaa. Latest consensus state: current_term: 1 leader_uuid: "4235617109b043efb90b0c6c066cccaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } } }
I20260810 14:55:44.888935 13065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [sys.catalog]: This master's current role is: LEADER
I20260810 14:55:44.889312 13068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:55:44.889688 13068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:55:44.890275 12863 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } attempt: 3
I20260810 14:55:44.892097 12995 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a [term 1 FOLLOWER]: Refusing update from remote peer 4235617109b043efb90b0c6c066cccaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:55:44.892457 13065 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:55:44.893784 13034 sys_catalog.cc:455] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4235617109b043efb90b0c6c066cccaa. Latest consensus state: current_term: 1 leader_uuid: "4235617109b043efb90b0c6c066cccaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } } }
I20260810 14:55:44.893891 13034 sys_catalog.cc:458] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:55:44.895413 13065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4235617109b043efb90b0c6c066cccaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } } }
I20260810 14:55:44.895505 13065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [sys.catalog]: This master's current role is: LEADER
Aug 10 14:55:44 dist-test-slave-0b58 krb5kdc[12776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373744, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for kudu/127.30.74.124@KRBTEST.COM
I20260810 14:55:44.895835 13034 sys_catalog.cc:455] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4235617109b043efb90b0c6c066cccaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } } }
I20260810 14:55:44.895936 13034 sys_catalog.cc:458] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:55:44.896003 13069 mvcc.cc:205] Tried to move back new op lower bound from 7316986859075366912 to 7316986859062788096. Current Snapshot: MvccSnapshot[applied={T|T < 7316986859075366912}]
I20260810 14:55:44.896607 13070 mvcc.cc:205] Tried to move back new op lower bound from 7316986859075366912 to 7316986859062788096. Current Snapshot: MvccSnapshot[applied={T|T < 7316986859075366912}]
I20260810 14:55:44.900440 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.890568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:34925 (local address 127.0.0.1:35378)
0810 14:55:44.890660 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:44.890683 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:44.890725 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:44.891073 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:44.891074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:44.891289 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:44.891344 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.891346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.892466 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.892467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.893238 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:44.893241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.894062 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.894063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:44.894142 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.894665 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:44.894677 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:44.896466 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:44.899111 (+ 2645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.899112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.899113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.899387 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.899646 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:44.899647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:44.899647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:44.899725 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:44.900054 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:44.900055 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:44.900128 (+ 73us) client_negotiation.cc:769] Sending connection context
0810 14:55:44.900193 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 14:55:44.900335 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 14:55:44.900843 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:44.890800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35378 (local address 127.30.74.124:34925)
0810 14:55:44.890844 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:44.890846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:44.890853 (+ 7us) server_negotiation.cc:407] Connection header received
0810 14:55:44.890876 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:44.890877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:44.890888 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:44.890935 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:44.891451 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.892231 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.893461 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:44.893781 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:44.893847 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:44.896586 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:44.896587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:44.896588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:44.896615 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:44.898456 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.899491 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.899492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.899493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.899567 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:44.899800 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:44.899801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:44.899801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:44.899955 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:44.900014 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:44.900680 (+ 666us) server_negotiation.cc:299] Negotiation successful
0810 14:55:44.900762 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 14:55:44.902827 12863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609: Bootstrap starting.
I20260810 14:55:44.903580 12863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609: Neither blocks nor log segments found. Creating new log.
I20260810 14:55:44.903890 12863 log.cc:824] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:44.905005 12863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609: No bootstrap required, opened a new log
I20260810 14:55:44.907020 12863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } }
I20260810 14:55:44.907157 12863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:55:44.907197 12863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 808ca1b71b46464ea4b0d1d500ac3609, State: Initialized, Role: FOLLOWER
I20260810 14:55:44.907382 13068 catalog_manager.cc:1347] Generated new cluster ID: 62e7ac139f5e4c09a53134778c235b8a
I20260810 14:55:44.907435 13068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:55:44.920388 12863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } }
I20260810 14:55:44.920874 12863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:55:44.921360 12834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4235617109b043efb90b0c6c066cccaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "808ca1b71b46464ea4b0d1d500ac3609" is_pre_election: true
I20260810 14:55:44.921522 12834 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4235617109b043efb90b0c6c066cccaa in term 0.
I20260810 14:55:44.921912 12832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4235617109b043efb90b0c6c066cccaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "808ca1b71b46464ea4b0d1d500ac3609"
I20260810 14:55:44.921993 12832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:55:44.922926 12832 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4235617109b043efb90b0c6c066cccaa in term 1.
I20260810 14:55:44.923341 12831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609 [term 1 FOLLOWER]: Refusing update from remote peer 4235617109b043efb90b0c6c066cccaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:55:44.923624 13051 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:55:44.925592 13088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4235617109b043efb90b0c6c066cccaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } } }
I20260810 14:55:44.925948 13088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:55:44.926182 13088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4235617109b043efb90b0c6c066cccaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } } }
I20260810 14:55:44.926308 13088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:55:44.925863 13075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4235617109b043efb90b0c6c066cccaa. Latest consensus state: current_term: 1 leader_uuid: "4235617109b043efb90b0c6c066cccaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } } }
I20260810 14:55:44.926440 13075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:55:44.929105 13068 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:55:44.929611 13089 mvcc.cc:205] Tried to move back new op lower bound from 7316986859075366912 to 7316986859062788096. Current Snapshot: MvccSnapshot[applied={T|T < 7316986859075366912}]
I20260810 14:55:44.929901 13068 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:55:44.933611 13092 catalog_manager.cc:1259] Loaded cluster ID: 62e7ac139f5e4c09a53134778c235b8a
I20260810 14:55:44.934175 13068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa: Generated new TSK 0
I20260810 14:55:44.934347 13068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:55:44.934563 13051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4235617109b043efb90b0c6c066cccaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808ca1b71b46464ea4b0d1d500ac3609" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43031 } } peers { permanent_uuid: "4235617109b043efb90b0c6c066cccaa" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 32785 } } peers { permanent_uuid: "a070e409243649c9a94fe74acf073d1a" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 34925 } } }
I20260810 14:55:44.934659 13051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4235617109b043efb90b0c6c066cccaa [sys.catalog]: This master's current role is: LEADER
I20260810 14:55:44.934929 13092 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609: loading cluster ID for follower catalog manager: success
W20260810 14:55:44.935102 13092 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260810 14:55:44.935473 13092 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:55:45.838850 13047 catalog_manager.cc:1259] Loaded cluster ID: 62e7ac139f5e4c09a53134778c235b8a
I20260810 14:55:45.838939 13047 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a: loading cluster ID for follower catalog manager: success
I20260810 14:55:45.840368 13047 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a: acquiring CA information for follower catalog manager: success
I20260810 14:55:45.840828 13047 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a070e409243649c9a94fe74acf073d1a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:55:45.939050 13092 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609: acquiring CA information for follower catalog manager: success
I20260810 14:55:45.939378 13092 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 808ca1b71b46464ea4b0d1d500ac3609: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:55:45.997426 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 12789
I20260810 14:55:46.005540 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 12870
I20260810 14:55:46.013196 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 12950
2026-08-10T14:55:46Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2975 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:55:46 dist-test-slave-0b58 krb5kdc[13105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:55:46 dist-test-slave-0b58 krb5kdc[13105](info): set up 2 sockets
Aug 10 14:55:46 dist-test-slave-0b58 krb5kdc[13105](info): commencing operation
krb5kdc: starting...
W20260810 14:55:47.130810 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:55:47 dist-test-slave-0b58 krb5kdc[13105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:55:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:55:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:55:47.278134 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:45233
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:45233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:55:47.371608 13118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:47.371925 13118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:47.371968 13118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:47.373445 13118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:55:47.373540 13118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:47.373564 13118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:47.373584 13118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:55:47.373615 13118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:55:47.375607 13118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:45233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:45233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:47.376001 13118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:47.376349 13118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:47.382396 13123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:47.382385 13126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:47.382416 13118 server_base.cc:1034] running on GCE node
W20260810 14:55:47.382391 13124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:47.382983 13118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:47.383293 13118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:47.384495 13118 hybrid_clock.cc:648] HybridClock initialized: now 1786373747384460 us; error 62 us; skew 500 ppm
Aug 10 14:55:47 dist-test-slave-0b58 krb5kdc[13105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373747, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:47.392882 13118 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:55:47.393481 13118 webserver.cc:460] Webserver started at http://127.30.74.126:40581/ using document root <none> and password file <none>
I20260810 14:55:47.393745 13118 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:47.393795 13118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:47.393961 13118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:55:47.395119 13118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "d017b223f48940399f0b3fa5342a1a73"
format_stamp: "Formatted at 2026-08-10 14:55:47 on dist-test-slave-0b58"
server_key: "9064850424348a5a3dd1a5fe13094af2"
server_key_iv: "9b017929f9341ae2e37e545727906361"
server_key_version: "encryptionkey@0"
I20260810 14:55:47.395539 13118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "d017b223f48940399f0b3fa5342a1a73"
format_stamp: "Formatted at 2026-08-10 14:55:47 on dist-test-slave-0b58"
server_key: "9064850424348a5a3dd1a5fe13094af2"
server_key_iv: "9b017929f9341ae2e37e545727906361"
server_key_version: "encryptionkey@0"
I20260810 14:55:47.397138 13118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:55:47.398002 13133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:47.398197 13118 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:47.398341 13118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "d017b223f48940399f0b3fa5342a1a73"
format_stamp: "Formatted at 2026-08-10 14:55:47 on dist-test-slave-0b58"
server_key: "9064850424348a5a3dd1a5fe13094af2"
server_key_iv: "9b017929f9341ae2e37e545727906361"
server_key_version: "encryptionkey@0"
I20260810 14:55:47.398428 13118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:47.411747 13118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:47.414283 13118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:47.414479 13118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:47.419701 13118 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:45233
I20260810 14:55:47.419741 13189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:45233 every 8 connection(s)
I20260810 14:55:47.420182 13118 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:55:47.421248 13190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:55:47.423892 13190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73: Bootstrap starting.
I20260810 14:55:47.424664 13190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73: Neither blocks nor log segments found. Creating new log.
I20260810 14:55:47.424991 13190 log.cc:824] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:47.425592 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 13118
I20260810 14:55:47.425714 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:55:47.425947 31017 external_mini_cluster.cc:1428] Setting key ba4eaf2e0e1ea07017fb8fd4392360d8
I20260810 14:55:47.426020 13190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73: No bootstrap required, opened a new log
I20260810 14:55:47.428030 13190 raft_consensus.cc:348] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d017b223f48940399f0b3fa5342a1a73" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45233 } }
I20260810 14:55:47.428207 13190 raft_consensus.cc:374] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:55:47.428231 13190 raft_consensus.cc:729] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d017b223f48940399f0b3fa5342a1a73, State: Initialized, Role: FOLLOWER
I20260810 14:55:47.428439 13190 consensus_queue.cc:260] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d017b223f48940399f0b3fa5342a1a73" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45233 } }
I20260810 14:55:47.428550 13190 raft_consensus.cc:388] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:55:47.428587 13190 raft_consensus.cc:482] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:55:47.428635 13190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:55:47.429798 13190 raft_consensus.cc:504] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d017b223f48940399f0b3fa5342a1a73" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45233 } }
I20260810 14:55:47.429910 13190 leader_election.cc:302] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d017b223f48940399f0b3fa5342a1a73; no voters:
I20260810 14:55:47.430130 13190 leader_election.cc:288] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:55:47.430189 13197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:55:47.430351 13197 raft_consensus.cc:686] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [term 1 LEADER]: Becoming Leader. State: Replica: d017b223f48940399f0b3fa5342a1a73, State: Running, Role: LEADER
I20260810 14:55:47.430445 13190 sys_catalog.cc:564] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:55:47.430445 13197 consensus_queue.cc:237] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d017b223f48940399f0b3fa5342a1a73" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45233 } }
I20260810 14:55:47.430979 13197 sys_catalog.cc:455] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d017b223f48940399f0b3fa5342a1a73. Latest consensus state: current_term: 1 leader_uuid: "d017b223f48940399f0b3fa5342a1a73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d017b223f48940399f0b3fa5342a1a73" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45233 } } }
I20260810 14:55:47.431063 13197 sys_catalog.cc:458] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [sys.catalog]: This master's current role is: LEADER
I20260810 14:55:47.432159 13198 sys_catalog.cc:455] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d017b223f48940399f0b3fa5342a1a73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d017b223f48940399f0b3fa5342a1a73" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45233 } } }
I20260810 14:55:47.432245 13198 sys_catalog.cc:458] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73 [sys.catalog]: This master's current role is: LEADER
W20260810 14:55:47.432600 13215 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:55:47.432653 13215 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:55:47.432711 13205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:55:47.432919 13205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 14:55:47 dist-test-slave-0b58 krb5kdc[13105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:47.434942 13205 catalog_manager.cc:1347] Generated new cluster ID: 4a9fc53238b34c3d8ea8ea532503d4ee
I20260810 14:55:47.434988 13205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:55:47.439728 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:47.426581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55944 (local address 127.30.74.126:45233)
0810 14:55:47.426830 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:47.426840 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:47.426872 (+ 32us) server_negotiation.cc:407] Connection header received
0810 14:55:47.428025 (+ 1153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:47.428035 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:47.428099 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:47.428342 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:47.429228 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:47.430453 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:47.431705 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:47.432437 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:47.432569 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:47.435609 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:47.435614 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:47.435614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:47.435628 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:47.437698 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:47.438112 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:47.438116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:47.438116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:47.438243 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:47.438450 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:47.438451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:47.438452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:47.438779 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:47.438912 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:47.439360 (+ 448us) server_negotiation.cc:299] Negotiation successful
0810 14:55:47.439505 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":155,"threads_started":1}
I20260810 14:55:47.440944 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:55:47.449615 13205 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:55:47.450223 13205 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:55:47.457572 13205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d017b223f48940399f0b3fa5342a1a73: Generated new TSK 0
I20260810 14:55:47.457862 13205 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:55:47.495695 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 13118
2026-08-10T14:55:47Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1481 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:55:47 dist-test-slave-0b58 krb5kdc[13221](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:55:47 dist-test-slave-0b58 krb5kdc[13221](info): set up 2 sockets
Aug 10 14:55:47 dist-test-slave-0b58 krb5kdc[13221](info): commencing operation
krb5kdc: starting...
W20260810 14:55:48.624038 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:55:48 dist-test-slave-0b58 krb5kdc[13221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:55:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:55:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:55:48.770417 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:45565
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:45565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:55:48.864830 13234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:48.865141 13234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:48.865197 13234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:48.866569 13234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:55:48.866663 13234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:48.866691 13234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:48.866715 13234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:55:48.866745 13234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:55:48.868571 13234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:45565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:45565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:48.868934 13234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:48.869221 13234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:48.875834 13240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:48.875820 13239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:48.875820 13242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:48.876168 13234 server_base.cc:1034] running on GCE node
I20260810 14:55:48.876541 13234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:48.876914 13234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:48.878149 13234 hybrid_clock.cc:648] HybridClock initialized: now 1786373748878191 us; error 95 us; skew 500 ppm
Aug 10 14:55:48 dist-test-slave-0b58 krb5kdc[13221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373748, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:48.882189 13234 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:55:48.883028 13234 webserver.cc:460] Webserver started at http://127.30.74.126:39373/ using document root <none> and password file <none>
I20260810 14:55:48.883414 13234 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:48.883513 13234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:48.883734 13234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:55:48.885435 13234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "49ae58c832d244d7aa6c2ef75d491ee6"
format_stamp: "Formatted at 2026-08-10 14:55:48 on dist-test-slave-0b58"
server_key: "697b38a47d3f3d802ed5d4eb5667d382"
server_key_iv: "a2f202b97c650beeef42a0c46a292dd3"
server_key_version: "encryptionkey@0"
I20260810 14:55:48.885924 13234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "49ae58c832d244d7aa6c2ef75d491ee6"
format_stamp: "Formatted at 2026-08-10 14:55:48 on dist-test-slave-0b58"
server_key: "697b38a47d3f3d802ed5d4eb5667d382"
server_key_iv: "a2f202b97c650beeef42a0c46a292dd3"
server_key_version: "encryptionkey@0"
I20260810 14:55:48.887562 13234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:55:48.888624 13249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:48.888912 13234 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:48.889041 13234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "49ae58c832d244d7aa6c2ef75d491ee6"
format_stamp: "Formatted at 2026-08-10 14:55:48 on dist-test-slave-0b58"
server_key: "697b38a47d3f3d802ed5d4eb5667d382"
server_key_iv: "a2f202b97c650beeef42a0c46a292dd3"
server_key_version: "encryptionkey@0"
I20260810 14:55:48.889132 13234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:48.904100 13234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:48.907152 13234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:48.907380 13234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:48.912170 13234 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:45565
I20260810 14:55:48.912278 13306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:45565 every 8 connection(s)
I20260810 14:55:48.912698 13234 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:55:48.913810 13307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:55:48.916503 13307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6: Bootstrap starting.
I20260810 14:55:48.917222 13307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6: Neither blocks nor log segments found. Creating new log.
I20260810 14:55:48.917483 13307 log.cc:824] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:48.917847 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 13234
I20260810 14:55:48.917945 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:55:48.918193 31017 external_mini_cluster.cc:1428] Setting key 4351128e571517aa04fffec17c4df9a8
I20260810 14:55:48.918470 13307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6: No bootstrap required, opened a new log
I20260810 14:55:48.920984 13307 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } }
I20260810 14:55:48.921182 13307 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:55:48.921233 13307 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49ae58c832d244d7aa6c2ef75d491ee6, State: Initialized, Role: FOLLOWER
I20260810 14:55:48.921377 13307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } }
I20260810 14:55:48.921481 13307 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:55:48.921528 13307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:55:48.921597 13307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:55:48.922256 13307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } }
I20260810 14:55:48.922392 13307 leader_election.cc:302] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49ae58c832d244d7aa6c2ef75d491ee6; no voters:
I20260810 14:55:48.922612 13307 leader_election.cc:288] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:55:48.922642 13312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:55:48.922766 13312 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 1 LEADER]: Becoming Leader. State: Replica: 49ae58c832d244d7aa6c2ef75d491ee6, State: Running, Role: LEADER
I20260810 14:55:48.922859 13312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } }
I20260810 14:55:48.922951 13307 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:55:48.923254 13314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } } }
I20260810 14:55:48.923369 13314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:55:48.923554 13313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49ae58c832d244d7aa6c2ef75d491ee6. Latest consensus state: current_term: 1 leader_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } } }
I20260810 14:55:48.923621 13313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:55:48.923763 13318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:55:48.924341 13318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:55:48.926815 13318 catalog_manager.cc:1347] Generated new cluster ID: c9f5502c07d148af8c7c85b1b8ca927f
I20260810 14:55:48.926891 13318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 14:55:48 dist-test-slave-0b58 krb5kdc[13221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:48.932613 13311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:48.918818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51642 (local address 127.30.74.126:45565)
0810 14:55:48.919044 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:48.919056 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:48.919094 (+ 38us) server_negotiation.cc:407] Connection header received
0810 14:55:48.920096 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:48.920106 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:48.920194 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:48.920402 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:48.921402 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:48.922946 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:48.924188 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:48.924845 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:48.924977 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:48.928191 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:48.928196 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:48.928197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:48.928211 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:48.930457 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:48.931003 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:48.931007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:48.931008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:48.931130 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:48.931384 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:48.931385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:48.931385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:48.931717 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:48.931874 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:48.932066 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 14:55:48.932378 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":100,"threads_started":1}
I20260810 14:55:48.934645 13318 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:55:48.935225 13318 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:55:48.944607 13318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6: Generated new TSK 0
I20260810 14:55:48.944880 13318 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:55:48.986584 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:45565
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:55:49.083323 13336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:49.083604 13336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:49.083634 13336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:49.085048 13336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:49.085114 13336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:49.085145 13336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:55:49.086941 13336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:49.087273 13336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:49.087564 13336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:49.093530 13342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:49.093518 13347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:49.093501 13343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:49.093845 13336 server_base.cc:1034] running on GCE node
I20260810 14:55:49.094195 13336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:49.094547 13336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:49.095736 13336 hybrid_clock.cc:648] HybridClock initialized: now 1786373749095704 us; error 61 us; skew 500 ppm
Aug 10 14:55:49 dist-test-slave-0b58 krb5kdc[13221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373749, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:49.098975 13336 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:55:49.099524 13336 webserver.cc:460] Webserver started at http://127.30.74.65:41527/ using document root <none> and password file <none>
I20260810 14:55:49.099781 13336 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:49.099869 13336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:49.100032 13336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:55:49.101327 13336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "35f5d65e079e47558d86a796107baddc"
format_stamp: "Formatted at 2026-08-10 14:55:49 on dist-test-slave-0b58"
server_key: "7c36203ca4564283d00cd1fbf6936fdf"
server_key_iv: "efd880da0c141979b0a773c7b41a9508"
server_key_version: "encryptionkey@0"
I20260810 14:55:49.101752 13336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "35f5d65e079e47558d86a796107baddc"
format_stamp: "Formatted at 2026-08-10 14:55:49 on dist-test-slave-0b58"
server_key: "7c36203ca4564283d00cd1fbf6936fdf"
server_key_iv: "efd880da0c141979b0a773c7b41a9508"
server_key_version: "encryptionkey@0"
I20260810 14:55:49.103295 13336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 14:55:49.104319 13355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:49.104525 13336 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:49.104648 13336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "35f5d65e079e47558d86a796107baddc"
format_stamp: "Formatted at 2026-08-10 14:55:49 on dist-test-slave-0b58"
server_key: "7c36203ca4564283d00cd1fbf6936fdf"
server_key_iv: "efd880da0c141979b0a773c7b41a9508"
server_key_version: "encryptionkey@0"
I20260810 14:55:49.104739 13336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:49.119920 13336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:49.122617 13336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:49.122869 13336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:49.123258 13336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:55:49.123714 13336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:55:49.123755 13336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:49.123780 13336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:55:49.123793 13336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:49.130498 13336 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:43907
I20260810 14:55:49.130540 13483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:43907 every 8 connection(s)
I20260810 14:55:49.130925 13336 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:55:49.134609 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 13336
I20260810 14:55:49.134800 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:55:49.135027 31017 external_mini_cluster.cc:1428] Setting key 561c0a168e7c68a9fa26fbd1dcb945f5
Aug 10 14:55:49 dist-test-slave-0b58 krb5kdc[13221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373749, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:49.143935 13311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.131886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:33129 (local address 127.30.74.126:45565)
0810 14:55:49.131973 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:49.131976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:49.132188 (+ 212us) server_negotiation.cc:407] Connection header received
0810 14:55:49.132789 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:49.132791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:49.132808 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:49.132885 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:49.134583 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.135457 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.136865 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.137201 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.137294 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.140158 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:49.140159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:49.140160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:49.140168 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:49.141960 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:49.142653 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:49.142654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:49.142655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:49.142719 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:49.143063 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:49.143064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:49.143064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:49.143198 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:49.143255 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:49.143746 (+ 491us) server_negotiation.cc:299] Negotiation successful
0810 14:55:49.143850 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 14:55:49.144132 13487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.131846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:33129)
0810 14:55:49.132038 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:49.132086 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:49.132643 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:49.132978 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:49.132984 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:49.133924 (+ 940us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:49.134387 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:49.134396 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.135588 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.135590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:49.136750 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:49.136754 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.137335 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.137335 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:49.137471 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.138243 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:49.138253 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:49.140018 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:49.142084 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:49.142098 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:49.142102 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:49.142534 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:49.142800 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:49.142801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:49.142802 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:49.142981 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:49.143275 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:49.143282 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:49.143503 (+ 221us) client_negotiation.cc:769] Sending connection context
0810 14:55:49.143642 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 14:55:49.143859 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
I20260810 14:55:49.145087 13484 heartbeater.cc:347] Connected to a master server at 127.30.74.126:45565
I20260810 14:55:49.145242 13484 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
I20260810 14:55:49.145470 13484 heartbeater.cc:511] Master 127.30.74.126:45565 requested a full tablet report, sending...
I20260810 14:55:49.146289 13267 ts_manager.cc:194] Registered new tserver with Master: 35f5d65e079e47558d86a796107baddc (127.30.74.65:43907)
I20260810 14:55:49.147037 13267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:33129
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:55:49.169648 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:45565
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:55:49.259117 13491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:49.259377 13491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:49.259407 13491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:49.260711 13491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:49.260761 13491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:49.260789 13491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:55:49.262369 13491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:49.262616 13491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:49.262831 13491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:49.268441 13496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:49.268464 13499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:49.268441 13497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:49.268774 13491 server_base.cc:1034] running on GCE node
I20260810 14:55:49.269052 13491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:49.269374 13491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:49.270565 13491 hybrid_clock.cc:648] HybridClock initialized: now 1786373749270552 us; error 52 us; skew 500 ppm
Aug 10 14:55:49 dist-test-slave-0b58 krb5kdc[13221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373749, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:49.273546 13491 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:55:49.274176 13491 webserver.cc:460] Webserver started at http://127.30.74.66:39925/ using document root <none> and password file <none>
I20260810 14:55:49.274559 13491 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:49.274631 13491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:49.274816 13491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:55:49.276225 13491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "37350b06c8e2459ea7ca5cc6ead6807c"
format_stamp: "Formatted at 2026-08-10 14:55:49 on dist-test-slave-0b58"
server_key: "f1d92d3595555febf61bad4cbcdba331"
server_key_iv: "3d675aea46a24c953f8e5dfcdd5c13e3"
server_key_version: "encryptionkey@0"
I20260810 14:55:49.276826 13491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "37350b06c8e2459ea7ca5cc6ead6807c"
format_stamp: "Formatted at 2026-08-10 14:55:49 on dist-test-slave-0b58"
server_key: "f1d92d3595555febf61bad4cbcdba331"
server_key_iv: "3d675aea46a24c953f8e5dfcdd5c13e3"
server_key_version: "encryptionkey@0"
I20260810 14:55:49.278390 13491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 14:55:49.279362 13507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:49.279726 13491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:49.279878 13491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "37350b06c8e2459ea7ca5cc6ead6807c"
format_stamp: "Formatted at 2026-08-10 14:55:49 on dist-test-slave-0b58"
server_key: "f1d92d3595555febf61bad4cbcdba331"
server_key_iv: "3d675aea46a24c953f8e5dfcdd5c13e3"
server_key_version: "encryptionkey@0"
I20260810 14:55:49.279976 13491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:49.290346 13491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:49.293220 13491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:49.293392 13491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:49.293715 13491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:55:49.294082 13491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:55:49.294148 13491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:49.294198 13491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:55:49.294222 13491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:49.300992 13491 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:43511
I20260810 14:55:49.301081 13633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:43511 every 8 connection(s)
I20260810 14:55:49.301463 13491 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:55:49.307363 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 13491
I20260810 14:55:49.307505 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:55:49.307763 31017 external_mini_cluster.cc:1428] Setting key dbf3071fbf7f75c1dc31876696f1891b
Aug 10 14:55:49 dist-test-slave-0b58 krb5kdc[13221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373749, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:49.313300 13311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.302508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:57695 (local address 127.30.74.126:45565)
0810 14:55:49.302593 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:49.302595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:49.302854 (+ 259us) server_negotiation.cc:407] Connection header received
0810 14:55:49.303462 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:49.303463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:49.303479 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:49.303540 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:49.304680 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.305226 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.306605 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.306844 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.306947 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.309662 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:49.309663 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:49.309663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:49.309673 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:49.311535 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:49.312070 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:49.312072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:49.312072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:49.312145 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:49.312513 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:49.312514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:49.312514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:49.312686 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:49.312769 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:49.313057 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 14:55:49.313162 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 14:55:49.313333 13636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.302491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:57695)
0810 14:55:49.302703 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:49.302746 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:49.303336 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:49.303624 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:49.303628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:49.304321 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:49.304562 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:49.304567 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.305363 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.305365 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:49.306464 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:49.306468 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.306959 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.306960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:49.307155 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.307918 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:49.307931 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:49.309535 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:49.311696 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:49.311705 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:49.311708 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:49.311982 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:49.312227 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:49.312228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:49.312228 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:49.312435 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:49.312767 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:49.312772 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:49.312910 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 14:55:49.312993 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 14:55:49.313142 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260810 14:55:49.314433 13634 heartbeater.cc:347] Connected to a master server at 127.30.74.126:45565
I20260810 14:55:49.314558 13634 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:49.314795 13634 heartbeater.cc:511] Master 127.30.74.126:45565 requested a full tablet report, sending...
I20260810 14:55:49.315310 13267 ts_manager.cc:194] Registered new tserver with Master: 37350b06c8e2459ea7ca5cc6ead6807c (127.30.74.66:43511)
I20260810 14:55:49.315773 13267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:57695
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:55:49.339797 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:45565
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:55:49.434902 13640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:49.435145 13640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:49.435173 13640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:49.436539 13640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:49.436590 13640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:49.436621 13640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:55:49.438282 13640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:49.438592 13640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:49.438894 13640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:49.444559 13648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:49.444756 13640 server_base.cc:1034] running on GCE node
W20260810 14:55:49.444579 13646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:49.444689 13645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:49.445109 13640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:49.445412 13640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:49.446578 13640 hybrid_clock.cc:648] HybridClock initialized: now 1786373749446560 us; error 36 us; skew 500 ppm
Aug 10 14:55:49 dist-test-slave-0b58 krb5kdc[13221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373749, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:49.449508 13640 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:55:49.450053 13640 webserver.cc:460] Webserver started at http://127.30.74.67:32985/ using document root <none> and password file <none>
I20260810 14:55:49.450340 13640 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:49.450415 13640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:49.450575 13640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:55:49.451719 13640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "f811229ee8704bc0bcfb7f059fd7a7dd"
format_stamp: "Formatted at 2026-08-10 14:55:49 on dist-test-slave-0b58"
server_key: "55aa4cf31d46256f8ae4d9514e759fb0"
server_key_iv: "548790f074319408f5ad27339f1caef1"
server_key_version: "encryptionkey@0"
I20260810 14:55:49.452124 13640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "f811229ee8704bc0bcfb7f059fd7a7dd"
format_stamp: "Formatted at 2026-08-10 14:55:49 on dist-test-slave-0b58"
server_key: "55aa4cf31d46256f8ae4d9514e759fb0"
server_key_iv: "548790f074319408f5ad27339f1caef1"
server_key_version: "encryptionkey@0"
I20260810 14:55:49.453691 13640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:55:49.454624 13655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:49.454876 13640 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:49.455003 13640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "f811229ee8704bc0bcfb7f059fd7a7dd"
format_stamp: "Formatted at 2026-08-10 14:55:49 on dist-test-slave-0b58"
server_key: "55aa4cf31d46256f8ae4d9514e759fb0"
server_key_iv: "548790f074319408f5ad27339f1caef1"
server_key_version: "encryptionkey@0"
I20260810 14:55:49.455091 13640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:49.463251 13640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:49.465726 13640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:49.465930 13640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:49.466212 13640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:55:49.466569 13640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:55:49.466627 13640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:49.466674 13640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:55:49.466698 13640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:49.473444 13640 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:37811
I20260810 14:55:49.473493 13782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:37811 every 8 connection(s)
I20260810 14:55:49.473876 13640 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:55:49.476966 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 13640
I20260810 14:55:49.477174 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:55:49.477428 31017 external_mini_cluster.cc:1428] Setting key 7f8066d9376c0f45a0cef37b645fb59a
Aug 10 14:55:49 dist-test-slave-0b58 krb5kdc[13221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373749, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:49.488169 13311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.474798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:57947 (local address 127.30.74.126:45565)
0810 14:55:49.474900 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:49.474903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:49.475223 (+ 320us) server_negotiation.cc:407] Connection header received
0810 14:55:49.475696 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:49.475698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:49.475710 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:49.475784 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:49.478073 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.478765 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.480613 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.480830 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.480902 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.484514 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:49.484516 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:49.484516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:49.484526 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:49.486057 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:49.486723 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:49.486725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:49.486725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:49.486796 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:49.487335 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:49.487336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:49.487337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:49.487481 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:49.487548 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:49.487994 (+ 446us) server_negotiation.cc:299] Negotiation successful
0810 14:55:49.488096 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 14:55:49.488386 13785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.474718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:57947)
0810 14:55:49.474909 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:49.474955 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:49.475547 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:49.475985 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:49.475990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:49.477278 (+ 1288us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:49.477643 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:49.477661 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.478894 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.478895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:49.480472 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:49.480476 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.480974 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.480975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:49.481116 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.481876 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:49.481886 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:49.484382 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:49.486189 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:49.486203 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:49.486206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:49.486532 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:49.486931 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:49.486932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:49.486933 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:49.487082 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:49.487582 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:49.487590 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:49.487779 (+ 189us) client_negotiation.cc:769] Sending connection context
0810 14:55:49.487886 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 14:55:49.488055 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":104,"threads_started":1}
I20260810 14:55:49.489063 13783 heartbeater.cc:347] Connected to a master server at 127.30.74.126:45565
I20260810 14:55:49.489256 13783 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:49.489568 13783 heartbeater.cc:511] Master 127.30.74.126:45565 requested a full tablet report, sending...
I20260810 14:55:49.490046 13267 ts_manager.cc:194] Registered new tserver with Master: f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811)
I20260810 14:55:49.490442 13267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:57947
I20260810 14:55:49.501484 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:55:49.510113 13311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.503072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51644 (local address 127.30.74.126:45565)
0810 14:55:49.503164 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:49.503166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:49.503287 (+ 121us) server_negotiation.cc:407] Connection header received
0810 14:55:49.503344 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:49.503345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:49.503355 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:49.503410 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:49.504106 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.504693 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.505471 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.505658 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.505715 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.507288 (+ 1573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:49.507289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:49.507290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:49.507297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:49.508892 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:49.509305 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:49.509306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:49.509306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:49.509369 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:49.509589 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:49.509589 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:49.509590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:49.509760 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:49.509842 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:49.509964 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 14:55:49.510040 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 14:55:49.511965 13267 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:55:49.513796 13267 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:55:49.521016 13796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.516568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:37811 (local address 127.0.0.1:43704)
0810 14:55:49.516963 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:49.516975 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:49.517044 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:49.517343 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:49.517346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:49.517350 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:55:49.517432 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:49.517434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.518437 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.518438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:49.519683 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:49.519686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.520672 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.520673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:49.520755 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.520772 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 14:55:49.520850 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 14:55:49.520917 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":43,"thread_start_us":98,"threads_started":1}
I20260810 14:55:49.521189 13795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.516564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:43511 (local address 127.0.0.1:39888)
0810 14:55:49.516893 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:49.516938 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:49.517043 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:49.517382 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:49.517384 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:49.517386 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:55:49.517446 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:49.517447 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.518400 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.518402 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:49.519826 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:49.519829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.520850 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.520852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:49.520918 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.520922 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:55:49.520976 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 14:55:49.521025 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":41,"thread_start_us":106,"threads_started":1}
I20260810 14:55:49.521245 13797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.516696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39888 (local address 127.30.74.66:43511)
0810 14:55:49.516876 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:49.516882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:49.517137 (+ 255us) server_negotiation.cc:407] Connection header received
0810 14:55:49.517188 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:49.517191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:49.517208 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:49.517283 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:55:49.517526 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.518294 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.519923 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.520660 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.520722 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.520768 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:49.521103 (+ 335us) server_negotiation.cc:299] Negotiation successful
0810 14:55:49.521159 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260810 14:55:49.521417 13798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.516758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43704 (local address 127.30.74.67:37811)
0810 14:55:49.517093 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:49.517097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:49.517108 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:55:49.517151 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:49.517153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:49.517170 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:49.517247 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:55:49.517538 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.518303 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.519775 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.520576 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.520671 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.520747 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:49.521275 (+ 528us) server_negotiation.cc:299] Negotiation successful
0810 14:55:49.521331 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":113,"threads_started":1}
I20260810 14:55:49.522085 13709 tablet_service.cc:1467] Processing CreateTablet for tablet 1b17c92d2aa445f2ad05daa7f0230e83 (DEFAULT_TABLE table=test-table [id=b8879b77bdfe4997a5fb5f542276130e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:55:49.522092 13560 tablet_service.cc:1467] Processing CreateTablet for tablet 1b17c92d2aa445f2ad05daa7f0230e83 (DEFAULT_TABLE table=test-table [id=b8879b77bdfe4997a5fb5f542276130e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:55:49.522433 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.516823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:43907 (local address 127.0.0.1:55140)
0810 14:55:49.518447 (+ 1624us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:49.518459 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:49.518518 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:49.518708 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:49.518709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:49.518711 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:55:49.518761 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:49.518762 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.519973 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.519974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:49.520845 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:49.520847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.522175 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.522176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:49.522241 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.522246 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:55:49.522306 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 14:55:49.522351 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1602,"thread_start_us":70,"threads_started":1}
I20260810 14:55:49.522490 13560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b17c92d2aa445f2ad05daa7f0230e83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:55:49.522490 13709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b17c92d2aa445f2ad05daa7f0230e83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:55:49.522693 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.517080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55140 (local address 127.30.74.65:43907)
0810 14:55:49.517304 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:49.517308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:49.518537 (+ 1229us) server_negotiation.cc:407] Connection header received
0810 14:55:49.518589 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:49.518592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:49.518615 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:49.518698 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:55:49.518823 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.519867 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.520934 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.522057 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.522170 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.522232 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:49.522569 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 14:55:49.522622 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":111,"threads_started":1}
I20260810 14:55:49.526654 13802 tablet_bootstrap.cc:492] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c: Bootstrap starting.
I20260810 14:55:49.527348 13409 tablet_service.cc:1467] Processing CreateTablet for tablet 1b17c92d2aa445f2ad05daa7f0230e83 (DEFAULT_TABLE table=test-table [id=b8879b77bdfe4997a5fb5f542276130e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:55:49.527496 13802 tablet_bootstrap.cc:654] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c: Neither blocks nor log segments found. Creating new log.
I20260810 14:55:49.527832 13409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b17c92d2aa445f2ad05daa7f0230e83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:55:49.528147 13802 log.cc:824] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:49.526327 13801 tablet_bootstrap.cc:492] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd: Bootstrap starting.
I20260810 14:55:49.529136 13802 tablet_bootstrap.cc:492] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c: No bootstrap required, opened a new log
I20260810 14:55:49.529239 13802 ts_tablet_manager.cc:1397] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 14:55:49.529294 13801 tablet_bootstrap.cc:654] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd: Neither blocks nor log segments found. Creating new log.
I20260810 14:55:49.529955 13801 log.cc:824] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:49.530917 13801 tablet_bootstrap.cc:492] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd: No bootstrap required, opened a new log
I20260810 14:55:49.531003 13801 ts_tablet_manager.cc:1397] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 14:55:49.531392 13802 raft_consensus.cc:348] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:49.531571 13802 raft_consensus.cc:374] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:55:49.531615 13802 raft_consensus.cc:729] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37350b06c8e2459ea7ca5cc6ead6807c, State: Initialized, Role: FOLLOWER
I20260810 14:55:49.531810 13802 consensus_queue.cc:260] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:49.532212 13802 ts_tablet_manager.cc:1428] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:55:49.532238 13634 heartbeater.cc:503] Master 127.30.74.126:45565 was elected leader, sending a full tablet report...
I20260810 14:55:49.532452 13806 tablet_bootstrap.cc:492] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: Bootstrap starting.
I20260810 14:55:49.533185 13806 tablet_bootstrap.cc:654] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: Neither blocks nor log segments found. Creating new log.
I20260810 14:55:49.533406 13801 raft_consensus.cc:348] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:49.533609 13801 raft_consensus.cc:374] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:55:49.533668 13801 raft_consensus.cc:729] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f811229ee8704bc0bcfb7f059fd7a7dd, State: Initialized, Role: FOLLOWER
I20260810 14:55:49.533805 13801 consensus_queue.cc:260] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:49.533843 13806 log.cc:824] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:49.534147 13801 ts_tablet_manager.cc:1428] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:55:49.534159 13783 heartbeater.cc:503] Master 127.30.74.126:45565 was elected leader, sending a full tablet report...
I20260810 14:55:49.534770 13806 tablet_bootstrap.cc:492] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: No bootstrap required, opened a new log
I20260810 14:55:49.534868 13806 ts_tablet_manager.cc:1397] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:55:49.536603 13806 raft_consensus.cc:348] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:49.536790 13806 raft_consensus.cc:374] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:55:49.536847 13806 raft_consensus.cc:729] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f5d65e079e47558d86a796107baddc, State: Initialized, Role: FOLLOWER
I20260810 14:55:49.536978 13806 consensus_queue.cc:260] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:49.537326 13806 ts_tablet_manager.cc:1428] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 14:55:49.537391 13484 heartbeater.cc:503] Master 127.30.74.126:45565 was elected leader, sending a full tablet report...
W20260810 14:55:49.552430 13635 tablet.cc:2369] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:55:49.632059 13485 tablet.cc:2369] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:55:49.724625 13784 tablet.cc:2369] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:55:49.730049 13810 raft_consensus.cc:482] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:55:49.730250 13810 raft_consensus.cc:504] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:49.730885 13810 leader_election.cc:288] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37350b06c8e2459ea7ca5cc6ead6807c (127.30.74.66:43511), f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811)
I20260810 14:55:49.735507 13798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.731309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:49205 (local address 127.30.74.67:37811)
0810 14:55:49.731372 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:49.731375 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:49.731387 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:55:49.731430 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:49.731431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:49.731441 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:49.731515 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:55:49.731733 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.732550 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.734328 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.734952 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.735049 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.735060 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:49.735368 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 14:55:49.735433 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 14:55:49.735500 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.730934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:43511 (local address 127.30.74.65:48681)
0810 14:55:49.731160 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:49.731173 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:49.731269 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:49.731484 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:49.731485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:49.731493 (+ 8us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:55:49.731589 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:49.731592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.732695 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.732697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:49.734152 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:49.734157 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.735109 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.735111 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:49.735226 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.735235 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 14:55:49.735317 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 14:55:49.735388 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":50,"thread_start_us":112,"threads_started":1}
I20260810 14:55:49.735502 13812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.730992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:37811 (local address 127.30.74.65:49205)
0810 14:55:49.731287 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:49.731300 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:49.731365 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:49.731515 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:49.731516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:49.731517 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:55:49.731616 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:49.731618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.732698 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.732698 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:49.734152 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:49.734157 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.735142 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.735143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:49.735226 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.735234 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 14:55:49.735317 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 14:55:49.735375 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":128,"thread_start_us":65,"threads_started":1}
I20260810 14:55:49.736171 13731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b17c92d2aa445f2ad05daa7f0230e83" candidate_uuid: "35f5d65e079e47558d86a796107baddc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" is_pre_election: true
I20260810 14:55:49.736469 13731 raft_consensus.cc:2454] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35f5d65e079e47558d86a796107baddc in term 0.
I20260810 14:55:49.736527 13797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.731003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:48681 (local address 127.30.74.66:43511)
0810 14:55:49.731083 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:49.731086 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:49.731235 (+ 149us) server_negotiation.cc:407] Connection header received
0810 14:55:49.731388 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:49.731389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:49.731402 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:49.731460 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:55:49.731733 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.732546 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.734320 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.734951 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.735049 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.735060 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:49.736335 (+ 1275us) server_negotiation.cc:299] Negotiation successful
0810 14:55:49.736426 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 14:55:49.736874 13358 leader_election.cc:302] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f5d65e079e47558d86a796107baddc, f811229ee8704bc0bcfb7f059fd7a7dd; no voters:
I20260810 14:55:49.737123 13810 raft_consensus.cc:2781] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:55:49.737193 13810 raft_consensus.cc:482] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:55:49.737128 13584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b17c92d2aa445f2ad05daa7f0230e83" candidate_uuid: "35f5d65e079e47558d86a796107baddc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" is_pre_election: true
I20260810 14:55:49.737237 13810 raft_consensus.cc:3037] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:55:49.737329 13584 raft_consensus.cc:2454] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35f5d65e079e47558d86a796107baddc in term 0.
I20260810 14:55:49.738339 13810 raft_consensus.cc:504] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:49.738523 13810 leader_election.cc:288] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 1 election: Requested vote from peers 37350b06c8e2459ea7ca5cc6ead6807c (127.30.74.66:43511), f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811)
I20260810 14:55:49.738729 13584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b17c92d2aa445f2ad05daa7f0230e83" candidate_uuid: "35f5d65e079e47558d86a796107baddc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37350b06c8e2459ea7ca5cc6ead6807c"
I20260810 14:55:49.738843 13584 raft_consensus.cc:3037] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:55:49.738821 13731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b17c92d2aa445f2ad05daa7f0230e83" candidate_uuid: "35f5d65e079e47558d86a796107baddc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd"
I20260810 14:55:49.738955 13731 raft_consensus.cc:3037] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:55:49.740352 13731 raft_consensus.cc:2454] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f5d65e079e47558d86a796107baddc in term 1.
I20260810 14:55:49.740023 13584 raft_consensus.cc:2454] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35f5d65e079e47558d86a796107baddc in term 1.
I20260810 14:55:49.740617 13358 leader_election.cc:302] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f5d65e079e47558d86a796107baddc, f811229ee8704bc0bcfb7f059fd7a7dd; no voters:
I20260810 14:55:49.740723 13810 raft_consensus.cc:2781] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:55:49.740918 13810 raft_consensus.cc:686] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [term 1 LEADER]: Becoming Leader. State: Replica: 35f5d65e079e47558d86a796107baddc, State: Running, Role: LEADER
I20260810 14:55:49.741037 13810 consensus_queue.cc:237] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:49.742050 13267 catalog_manager.cc:5495] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc reported cstate change: term changed from 0 to 1, leader changed from <none> to 35f5d65e079e47558d86a796107baddc (127.30.74.65). New cstate: current_term: 1 leader_uuid: "35f5d65e079e47558d86a796107baddc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } health_report { overall_health: HEALTHY } } }
I20260810 14:55:49.755863 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.753235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55150 (local address 127.30.74.65:43907)
0810 14:55:49.753318 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:49.753326 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:49.753340 (+ 14us) server_negotiation.cc:407] Connection header received
0810 14:55:49.753383 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:49.753384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:49.753404 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:49.753498 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:55:49.753688 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.754184 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.754999 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:49.755209 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:49.755292 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:49.755498 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:55:49.755617 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:55:49.755720 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:49.755741 (+ 21us) server_negotiation.cc:299] Negotiation successful
0810 14:55:49.755764 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 14:55:49.758167 13584 raft_consensus.cc:1261] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c [term 1 FOLLOWER]: Refusing update from remote peer 35f5d65e079e47558d86a796107baddc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:55:49.758203 13731 raft_consensus.cc:1261] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd [term 1 FOLLOWER]: Refusing update from remote peer 35f5d65e079e47558d86a796107baddc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:55:49.758607 13813 consensus_queue.cc:1035] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [LEADER]: Connected to new peer: Peer: permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:55:49.758795 13810 consensus_queue.cc:1035] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:55:49.761511 13816 mvcc.cc:205] Tried to move back new op lower bound from 7316986879005896704 to 7316986878940004352. Current Snapshot: MvccSnapshot[applied={T|T < 7316986879005896704}]
I20260810 14:55:49.761931 13818 mvcc.cc:205] Tried to move back new op lower bound from 7316986879005896704 to 7316986878940004352. Current Snapshot: MvccSnapshot[applied={T|T < 7316986879005896704}]
I20260810 14:55:49.764331 13267 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:55:49.764446 13267 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:55:49.765441 13267 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6: Sending DeleteTablet for 3 replicas of tablet 1b17c92d2aa445f2ad05daa7f0230e83
I20260810 14:55:49.765887 13560 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b17c92d2aa445f2ad05daa7f0230e83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:55:49 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:39888
I20260810 14:55:49.765903 13709 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b17c92d2aa445f2ad05daa7f0230e83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:55:49 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:43704
I20260810 14:55:49.766183 13409 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b17c92d2aa445f2ad05daa7f0230e83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:55:49 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:55140
I20260810 14:55:49.766321 13828 tablet_replica.cc:330] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd: stopping tablet replica
I20260810 14:55:49.766335 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 13336
I20260810 14:55:49.768762 13829 tablet_replica.cc:330] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c: stopping tablet replica
I20260810 14:55:49.768954 13829 raft_consensus.cc:2229] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:55:49.769133 13829 raft_consensus.cc:2258] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:55:49.769484 13828 raft_consensus.cc:2229] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:55:49.769654 13828 raft_consensus.cc:2258] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:55:49.770295 13828 ts_tablet_manager.cc:1905] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:55:49.772632 13828 ts_tablet_manager.cc:1918] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:55:49.772743 13828 log.cc:1196] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/1b17c92d2aa445f2ad05daa7f0230e83
I20260810 14:55:49.772969 13829 ts_tablet_manager.cc:1905] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:55:49.773080 13828 ts_tablet_manager.cc:1939] T 1b17c92d2aa445f2ad05daa7f0230e83 P f811229ee8704bc0bcfb7f059fd7a7dd: Deleting consensus metadata
I20260810 14:55:49.773643 13252 catalog_manager.cc:4841] TS f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811): tablet 1b17c92d2aa445f2ad05daa7f0230e83 (table test-table [id=b8879b77bdfe4997a5fb5f542276130e]) successfully deleted
W20260810 14:55:49.774788 13250 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:43907 (error 108)
I20260810 14:55:49.775020 13829 ts_tablet_manager.cc:1918] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:55:49.775080 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 13491
I20260810 14:55:49.775110 13829 log.cc:1196] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/1b17c92d2aa445f2ad05daa7f0230e83
I20260810 14:55:49.775398 13829 ts_tablet_manager.cc:1939] T 1b17c92d2aa445f2ad05daa7f0230e83 P 37350b06c8e2459ea7ca5cc6ead6807c: Deleting consensus metadata
I20260810 14:55:49.777249 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.777074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:43907 (local address 127.0.0.1:55160)
0810 14:55:49.777153 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:49.777206 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:43907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 14:55:49.777400 13250 catalog_manager.cc:4584] TS 35f5d65e079e47558d86a796107baddc (127.30.74.65:43907): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1b17c92d2aa445f2ad05daa7f0230e83: Network error: Client connection negotiation failed: client connection to 127.30.74.65:43907: connect: Connection refused (error 111)
I20260810 14:55:49.783422 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 13640
I20260810 14:55:49.785419 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:49.783534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:43511 (local address 127.0.0.1:39896)
0810 14:55:49.785315 (+ 1781us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:49.785363 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.66:43511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1744}
I20260810 14:55:49.794761 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 13234
I20260810 14:55:49.803962 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:45565
--webserver_interface=127.30.74.126
--webserver_port=39373
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:45565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:55:49.897930 13833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:49.898257 13833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:49.898288 13833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:49.899672 13833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:55:49.899739 13833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:49.899753 13833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:49.899766 13833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:55:49.899780 13833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:55:49.901530 13833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:45565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:45565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=39373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:49.901929 13833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:49.902177 13833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:49.913127 13840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:49.913012 13845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:49.913024 13842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:49.913048 13833 server_base.cc:1034] running on GCE node
I20260810 14:55:49.913647 13833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:49.914100 13833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:49.915318 13833 hybrid_clock.cc:648] HybridClock initialized: now 1786373749915295 us; error 36 us; skew 500 ppm
Aug 10 14:55:49 dist-test-slave-0b58 krb5kdc[13221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373749, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:49.918794 13833 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:55:49.919438 13833 webserver.cc:460] Webserver started at http://127.30.74.126:39373/ using document root <none> and password file <none>
I20260810 14:55:49.919759 13833 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:49.919871 13833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:49.921849 13833 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:49.922981 13852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:49.923241 13833 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:49.923372 13833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "49ae58c832d244d7aa6c2ef75d491ee6"
format_stamp: "Formatted at 2026-08-10 14:55:48 on dist-test-slave-0b58"
server_key: "697b38a47d3f3d802ed5d4eb5667d382"
server_key_iv: "a2f202b97c650beeef42a0c46a292dd3"
server_key_version: "encryptionkey@0"
I20260810 14:55:49.923872 13833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:49.941290 13833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:49.944396 13833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:49.944550 13833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:49.949138 13833 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:45565
I20260810 14:55:49.949270 13909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:45565 every 8 connection(s)
I20260810 14:55:49.949826 13833 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:55:49.951812 13910 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:55:49.952024 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 13833
I20260810 14:55:49.952440 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:43907
--local_ip_for_outbound_sockets=127.30.74.65
--tserver_master_addrs=127.30.74.126:45565
--webserver_port=41527
--webserver_interface=127.30.74.65
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:55:49.952901 13910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6: Bootstrap starting.
I20260810 14:55:49.962947 13910 log.cc:824] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:49.965452 13910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:55:49.965929 13910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6: Bootstrap complete.
I20260810 14:55:49.968664 13910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } }
I20260810 14:55:49.969079 13910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49ae58c832d244d7aa6c2ef75d491ee6, State: Initialized, Role: FOLLOWER
I20260810 14:55:49.969380 13910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } }
I20260810 14:55:49.969509 13910 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:55:49.969605 13910 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:55:49.969663 13910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:55:49.971037 13910 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } }
I20260810 14:55:49.971200 13910 leader_election.cc:302] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49ae58c832d244d7aa6c2ef75d491ee6; no voters:
I20260810 14:55:49.971478 13910 leader_election.cc:288] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 14:55:49.971529 13915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 14:55:49.971657 13915 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 2 LEADER]: Becoming Leader. State: Replica: 49ae58c832d244d7aa6c2ef75d491ee6, State: Running, Role: LEADER
I20260810 14:55:49.971804 13910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:55:49.971776 13915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } }
I20260810 14:55:49.972128 13915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49ae58c832d244d7aa6c2ef75d491ee6. Latest consensus state: current_term: 2 leader_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } } }
I20260810 14:55:49.972200 13915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:55:49.972434 13920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:55:49.972532 13916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } } }
I20260810 14:55:49.972721 13916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:55:49.973443 13920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:55:49.973587 13920 catalog_manager.cc:1259] Loaded cluster ID: c9f5502c07d148af8c7c85b1b8ca927f
I20260810 14:55:49.973843 13920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:55:49.974138 13920 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 14:55:49.979243 13920 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 14:55:49.982539 13920 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:55:49.982859 13920 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6: Loaded TSK: 0
I20260810 14:55:49.983131 13920 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 14:55:50.048039 13912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:50.048362 13912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:50.048409 13912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:50.049829 13912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:50.049913 13912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:50.049964 13912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:55:50.051640 13912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:43907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=41527
--tserver_master_addrs=127.30.74.126:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:50.051955 13912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:50.052278 13912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:50.057886 13940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:50.057896 13939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:50.057896 13942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:50.058888 13912 server_base.cc:1034] running on GCE node
I20260810 14:55:50.059159 13912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:50.059504 13912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:50.060711 13912 hybrid_clock.cc:648] HybridClock initialized: now 1786373750060682 us; error 36 us; skew 500 ppm
Aug 10 14:55:50 dist-test-slave-0b58 krb5kdc[13221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373750, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:50.063705 13912 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:55:50.064231 13912 webserver.cc:460] Webserver started at http://127.30.74.65:41527/ using document root <none> and password file <none>
I20260810 14:55:50.064550 13912 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:50.064618 13912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:50.066288 13912 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:55:50.067211 13949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:50.067468 13912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:50.067584 13912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "35f5d65e079e47558d86a796107baddc"
format_stamp: "Formatted at 2026-08-10 14:55:49 on dist-test-slave-0b58"
server_key: "7c36203ca4564283d00cd1fbf6936fdf"
server_key_iv: "efd880da0c141979b0a773c7b41a9508"
server_key_version: "encryptionkey@0"
I20260810 14:55:50.067996 13912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:50.101392 13912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:50.104328 13912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:50.104526 13912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:50.104830 13912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:55:50.105351 13956 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 14:55:50.107645 13912 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 14:55:50.107746 13912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260810 14:55:50.107805 13912 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 14:55:50.108711 13912 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 14:55:50.108774 13912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:50.108842 13956 tablet_bootstrap.cc:492] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: Bootstrap starting.
I20260810 14:55:50.116453 13912 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:43907
I20260810 14:55:50.116468 14084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:43907 every 8 connection(s)
I20260810 14:55:50.116917 13912 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:55:50.120085 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 13912
I20260810 14:55:50.120472 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:43511
--local_ip_for_outbound_sockets=127.30.74.66
--tserver_master_addrs=127.30.74.126:45565
--webserver_port=39925
--webserver_interface=127.30.74.66
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:55:50.121223 13956 log.cc:824] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:50.124538 13956 tablet_bootstrap.cc:492] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:55:50.124993 13956 tablet_bootstrap.cc:492] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: Bootstrap complete.
I20260810 14:55:50.126179 13956 ts_tablet_manager.cc:1397] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.013s
Aug 10 14:55:50 dist-test-slave-0b58 krb5kdc[13221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373750, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:50.127823 13956 raft_consensus.cc:348] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:50.128252 13956 raft_consensus.cc:729] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f5d65e079e47558d86a796107baddc, State: Initialized, Role: FOLLOWER
I20260810 14:55:50.128479 13956 consensus_queue.cc:260] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:50.128867 13956 ts_tablet_manager.cc:1428] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260810 14:55:50.131704 14087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.117753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:40457)
0810 14:55:50.117987 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:50.118044 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:50.118585 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:50.119074 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:50.119078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:50.119389 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:50.119593 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:50.119598 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.121336 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.121338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:50.122661 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:50.122665 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.123620 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.123622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:50.123815 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.124501 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:50.124512 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:50.126819 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:50.129820 (+ 3001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:50.129827 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:50.129830 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:50.130092 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:50.130508 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:50.130510 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:50.130510 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:50.130630 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:50.131170 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:50.131175 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:50.131306 (+ 131us) client_negotiation.cc:769] Sending connection context
0810 14:55:50.131396 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 14:55:50.131547 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":114,"threads_started":1}
I20260810 14:55:50.131924 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.117847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:40457 (local address 127.30.74.126:45565)
0810 14:55:50.118078 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:50.118088 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:50.118131 (+ 43us) server_negotiation.cc:407] Connection header received
0810 14:55:50.118847 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:50.118852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:50.118899 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:50.119034 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:50.119724 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.121178 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.122776 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.123473 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.123600 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.126981 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:50.126987 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:50.126988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:50.127006 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:50.129690 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:50.130237 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:50.130241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:50.130242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:50.130402 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:50.130716 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:50.130717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:50.130718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:50.131089 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:50.131189 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:50.131498 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 14:55:50.131678 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":114,"threads_started":1}
I20260810 14:55:50.132735 14085 heartbeater.cc:347] Connected to a master server at 127.30.74.126:45565
I20260810 14:55:50.132864 14085 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:50.133100 14085 heartbeater.cc:511] Master 127.30.74.126:45565 requested a full tablet report, sending...
I20260810 14:55:50.134007 13873 ts_manager.cc:194] Registered new tserver with Master: 35f5d65e079e47558d86a796107baddc (127.30.74.65:43907)
I20260810 14:55:50.135095 13873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:40457
I20260810 14:55:50.136135 14085 heartbeater.cc:503] Master 127.30.74.126:45565 was elected leader, sending a full tablet report...
Aug 10 14:55:50 dist-test-slave-0b58 krb5kdc[13221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373749, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for kudu/127.30.74.65@KRBTEST.COM
I20260810 14:55:50.145535 14093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.134675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:43907 (local address 127.0.0.1:55170)
0810 14:55:50.134887 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:50.134928 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:50.135000 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:50.135799 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:50.135806 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:50.136172 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:50.136243 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:50.136246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.137674 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.137676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:50.138401 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:50.138404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.139125 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.139126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:50.139235 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.139931 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:50.139937 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:50.141771 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:50.144028 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:50.144039 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:50.144041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:50.144395 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:50.144695 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:50.144696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:50.144696 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:50.144778 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:50.145161 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:50.145169 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:50.145225 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 14:55:50.145306 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 14:55:50.145437 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":114,"threads_started":1}
I20260810 14:55:50.145587 14094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.135401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55170 (local address 127.30.74.65:43907)
0810 14:55:50.135618 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:50.135623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:50.135634 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:55:50.135674 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:50.135676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:50.135691 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:50.135769 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:50.136393 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.137529 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.138496 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.138988 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.139072 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.141883 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:50.141897 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:50.141898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:50.141907 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:50.143899 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:50.144494 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:50.144498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:50.144498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:50.144576 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:50.144871 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:50.144872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:50.144872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:50.145081 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:50.145166 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:50.145381 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 14:55:50.145481 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":131,"threads_started":1}
I20260810 14:55:50.146224 14009 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b17c92d2aa445f2ad05daa7f0230e83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:55:49 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:55170
I20260810 14:55:50.146909 14095 tablet_replica.cc:330] stopping tablet replica
I20260810 14:55:50.147053 14095 raft_consensus.cc:2229] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:55:50.147145 14095 raft_consensus.cc:2258] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:55:50.147533 14095 ts_tablet_manager.cc:1905] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:55:50.150063 14095 ts_tablet_manager.cc:1918] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:55:50.150205 14095 log.cc:1196] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal/wals/1b17c92d2aa445f2ad05daa7f0230e83
I20260810 14:55:50.150484 14095 ts_tablet_manager.cc:1939] T 1b17c92d2aa445f2ad05daa7f0230e83 P 35f5d65e079e47558d86a796107baddc: Deleting consensus metadata
I20260810 14:55:50.151160 13853 catalog_manager.cc:4841] TS 35f5d65e079e47558d86a796107baddc (127.30.74.65:43907): tablet 1b17c92d2aa445f2ad05daa7f0230e83 (table test-table [id=b8879b77bdfe4997a5fb5f542276130e]) successfully deleted
W20260810 14:55:50.221897 14090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:50.222143 14090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:50.222172 14090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:50.224507 14090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:50.224612 14090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:50.224682 14090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:55:50.227766 14090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:43511
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=39925
--tserver_master_addrs=127.30.74.126:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:50.228175 14090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:50.228550 14090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:50.235185 14104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:50.235234 14102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:50.235160 14101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:50.235917 14090 server_base.cc:1034] running on GCE node
I20260810 14:55:50.236230 14090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:50.236594 14090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:50.237819 14090 hybrid_clock.cc:648] HybridClock initialized: now 1786373750237796 us; error 34 us; skew 500 ppm
Aug 10 14:55:50 dist-test-slave-0b58 krb5kdc[13221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373750, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:50.241036 14090 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:55:50.241596 14090 webserver.cc:460] Webserver started at http://127.30.74.66:39925/ using document root <none> and password file <none>
I20260810 14:55:50.241890 14090 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:50.241953 14090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:50.243629 14090 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:55:50.244491 14113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:50.244693 14090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:50.244799 14090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "37350b06c8e2459ea7ca5cc6ead6807c"
format_stamp: "Formatted at 2026-08-10 14:55:49 on dist-test-slave-0b58"
server_key: "f1d92d3595555febf61bad4cbcdba331"
server_key_iv: "3d675aea46a24c953f8e5dfcdd5c13e3"
server_key_version: "encryptionkey@0"
I20260810 14:55:50.245199 14090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:50.259433 14090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:50.262378 14090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:50.262526 14090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:50.262759 14090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:55:50.263150 14090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:55:50.263195 14090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:50.263221 14090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:55:50.263275 14090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:50.270303 14090 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:43511
I20260810 14:55:50.270388 14236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:43511 every 8 connection(s)
I20260810 14:55:50.270810 14090 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
Aug 10 14:55:50 dist-test-slave-0b58 krb5kdc[13221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373750, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:50.279271 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 14090
I20260810 14:55:50.279569 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:37811
--local_ip_for_outbound_sockets=127.30.74.67
--tserver_master_addrs=127.30.74.126:45565
--webserver_port=32985
--webserver_interface=127.30.74.67
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:55:50.282461 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.271663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:58237 (local address 127.30.74.126:45565)
0810 14:55:50.271758 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:50.271760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:50.272022 (+ 262us) server_negotiation.cc:407] Connection header received
0810 14:55:50.272684 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:50.272685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:50.272697 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:50.272764 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:50.273885 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.274402 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.275531 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.275745 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.275810 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.278585 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:50.278586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:50.278587 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:50.278596 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:50.280888 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:50.281383 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:50.281384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:50.281385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:50.281454 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:50.281735 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:50.281736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:50.281736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:50.281869 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:50.281952 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:50.282257 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 14:55:50.282365 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 14:55:50.282744 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.271672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:58237)
0810 14:55:50.271868 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:50.271914 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:50.272563 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:50.272847 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:50.272850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:50.273556 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:50.273769 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:50.273775 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.274515 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.274516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:50.275421 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:50.275424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.275853 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.275854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:50.275987 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.276669 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:50.276679 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:50.278441 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:50.280997 (+ 2556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:50.281009 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:50.281012 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:50.281293 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:50.281530 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:50.281531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:50.281532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:50.281663 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:50.281942 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:50.281950 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:50.282107 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 14:55:50.282225 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 14:55:50.282411 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":92,"threads_started":1}
I20260810 14:55:50.283378 14238 heartbeater.cc:347] Connected to a master server at 127.30.74.126:45565
I20260810 14:55:50.283509 14238 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:50.283756 14238 heartbeater.cc:511] Master 127.30.74.126:45565 requested a full tablet report, sending...
I20260810 14:55:50.284324 13873 ts_manager.cc:194] Registered new tserver with Master: 37350b06c8e2459ea7ca5cc6ead6807c (127.30.74.66:43511)
I20260810 14:55:50.284883 13873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:58237
W20260810 14:55:50.369015 14241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:50.369277 14241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:50.369307 14241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:50.370705 14241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:50.370781 14241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:50.370814 14241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:55:50.373077 14241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:37811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=32985
--tserver_master_addrs=127.30.74.126:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:50.373387 14241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:50.373647 14241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:50.380087 14249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:50.380082 14247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:50.380142 14246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:50.380474 14241 server_base.cc:1034] running on GCE node
I20260810 14:55:50.380935 14241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:50.381273 14241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:50.382467 14241 hybrid_clock.cc:648] HybridClock initialized: now 1786373750382435 us; error 51 us; skew 500 ppm
Aug 10 14:55:50 dist-test-slave-0b58 krb5kdc[13221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373750, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:50.385317 14241 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:55:50.385895 14241 webserver.cc:460] Webserver started at http://127.30.74.67:32985/ using document root <none> and password file <none>
I20260810 14:55:50.386159 14241 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:50.386226 14241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:50.387843 14241 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:55:50.388724 14256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:50.388957 14241 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:50.389060 14241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "f811229ee8704bc0bcfb7f059fd7a7dd"
format_stamp: "Formatted at 2026-08-10 14:55:49 on dist-test-slave-0b58"
server_key: "55aa4cf31d46256f8ae4d9514e759fb0"
server_key_iv: "548790f074319408f5ad27339f1caef1"
server_key_version: "encryptionkey@0"
I20260810 14:55:50.389441 14241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:50.402096 14241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:50.404829 14241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:50.404968 14241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:50.405182 14241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:55:50.405578 14241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:55:50.405617 14241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:50.405640 14241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:55:50.405653 14241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:50.412010 14241 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:37811
I20260810 14:55:50.412107 14385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:37811 every 8 connection(s)
I20260810 14:55:50.412519 14241 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:55:50.417475 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 14241
Aug 10 14:55:50 dist-test-slave-0b58 krb5kdc[13221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373750, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:55:50.424860 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.413407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:59457 (local address 127.30.74.126:45565)
0810 14:55:50.413504 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:50.413507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:50.413776 (+ 269us) server_negotiation.cc:407] Connection header received
0810 14:55:50.414420 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:50.414422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:50.414438 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:50.414500 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:50.415701 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.416371 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.417717 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.418079 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.418165 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.421255 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:50.421256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:50.421257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:50.421266 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:50.422868 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:50.423487 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:50.423489 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:50.423490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:50.423573 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:50.423983 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:50.423985 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:50.423985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:50.424158 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:50.424250 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:50.424697 (+ 447us) server_negotiation.cc:299] Negotiation successful
0810 14:55:50.424787 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 14:55:50.425525 14388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.413370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:59457)
0810 14:55:50.413592 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:50.413636 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:50.414286 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:50.414599 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:50.414603 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:50.415285 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:50.415547 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:50.415553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.416512 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.416513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:50.417538 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:50.417560 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.418192 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.418193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:50.418442 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.419113 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:50.419122 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:50.421129 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:50.422995 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:50.423003 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:50.423023 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:50.423395 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:50.423673 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:50.423674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:50.423674 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:50.423877 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:50.424368 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:50.424376 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:50.424537 (+ 161us) client_negotiation.cc:769] Sending connection context
0810 14:55:50.424646 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 14:55:50.425204 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":100,"threads_started":1}
I20260810 14:55:50.426113 14390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.418060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51658 (local address 127.30.74.126:45565)
0810 14:55:50.418245 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:50.418247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:50.418279 (+ 32us) server_negotiation.cc:407] Connection header received
0810 14:55:50.418379 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:50.418380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:50.418393 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:50.418476 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:50.419278 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.420008 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.421222 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.421454 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.421552 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.422843 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:50.422845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:50.422846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:50.422853 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:50.424602 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:50.425089 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:50.425090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:50.425090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:50.425141 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:50.425408 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:50.425409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:50.425409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:50.425513 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:50.425686 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:50.425867 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 14:55:50.425965 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":108,"threads_started":1}
I20260810 14:55:50.426159 14386 heartbeater.cc:347] Connected to a master server at 127.30.74.126:45565
I20260810 14:55:50.426280 14386 heartbeater.cc:464] Registering TS with master...
I20260810 14:55:50.426563 14386 heartbeater.cc:511] Master 127.30.74.126:45565 requested a full tablet report, sending...
I20260810 14:55:50.427079 13873 ts_manager.cc:194] Registered new tserver with Master: f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811)
I20260810 14:55:50.427443 13873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:59457
I20260810 14:55:50.438383 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:55:50.441784 14390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.438879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51674 (local address 127.30.74.126:45565)
0810 14:55:50.438963 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:50.438970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:50.439058 (+ 88us) server_negotiation.cc:407] Connection header received
0810 14:55:50.439107 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:50.439108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:50.439150 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:50.439229 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:55:50.439366 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.439872 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.440873 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.441081 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.441131 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.441335 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:55:50.441451 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:55:50.441530 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:50.441610 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 14:55:50.441636 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 14:55:50.442173 13873 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:51674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:55:50.443977 13873 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:55:50.447281 14009 tablet_service.cc:1467] Processing CreateTablet for tablet 5dbc11b9268945ca9f4e18f3d83fd477 (DEFAULT_TABLE table=test-table [id=adc7c87081df4c50a3eba418fee58a0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:55:50.447592 14009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dbc11b9268945ca9f4e18f3d83fd477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:55:50.449412 13956 tablet_bootstrap.cc:492] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc: Bootstrap starting.
I20260810 14:55:50.449784 13956 tablet_bootstrap.cc:654] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc: Neither blocks nor log segments found. Creating new log.
I20260810 14:55:50.450798 13956 tablet_bootstrap.cc:492] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc: No bootstrap required, opened a new log
I20260810 14:55:50.450862 13956 ts_tablet_manager.cc:1397] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 14:55:50.451016 13956 raft_consensus.cc:348] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:50.451097 13956 raft_consensus.cc:374] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:55:50.451141 13956 raft_consensus.cc:729] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f5d65e079e47558d86a796107baddc, State: Initialized, Role: FOLLOWER
I20260810 14:55:50.451190 13956 consensus_queue.cc:260] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:50.451331 13956 ts_tablet_manager.cc:1428] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 14:55:50.452046 14093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.446744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:37811 (local address 127.0.0.1:43714)
0810 14:55:50.446874 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:50.446888 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:50.446946 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:50.447661 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:50.447662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:50.447666 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:55:50.447733 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:50.447735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.449188 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.449189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:50.450640 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:50.450643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.451735 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.451736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:50.451813 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.451845 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 14:55:50.451910 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 14:55:50.451968 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260810 14:55:50.452064 14393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.446942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43714 (local address 127.30.74.67:37811)
0810 14:55:50.447392 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:50.447397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:50.447410 (+ 13us) server_negotiation.cc:407] Connection header received
0810 14:55:50.447462 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:50.447464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:50.447482 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:50.447621 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:55:50.447898 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.449047 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.450748 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.451640 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.451736 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.451816 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:50.451929 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 14:55:50.451972 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":94,"threads_started":1}
I20260810 14:55:50.453136 14311 tablet_service.cc:1467] Processing CreateTablet for tablet 5dbc11b9268945ca9f4e18f3d83fd477 (DEFAULT_TABLE table=test-table [id=adc7c87081df4c50a3eba418fee58a0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:55:50.453286 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.446744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:43511 (local address 127.0.0.1:39908)
0810 14:55:50.447042 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:50.447052 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:50.448329 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:50.448593 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:50.448594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:50.448597 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:55:50.448648 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:50.448650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.450500 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.450501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:50.451765 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:50.451768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.452994 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.452995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:50.453073 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.453078 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:55:50.453143 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 14:55:50.453197 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":78,"thread_start_us":89,"threads_started":1}
I20260810 14:55:50.453296 14394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.447076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39908 (local address 127.30.74.66:43511)
0810 14:55:50.447389 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:50.447394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:50.447406 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:55:50.448465 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:50.448469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:50.448487 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:50.448596 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:55:50.448725 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.450356 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.451874 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.452887 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.452991 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.453056 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:50.453165 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 14:55:50.453210 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":122,"threads_started":1}
I20260810 14:55:50.453569 14311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dbc11b9268945ca9f4e18f3d83fd477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:55:50.454085 14167 tablet_service.cc:1467] Processing CreateTablet for tablet 5dbc11b9268945ca9f4e18f3d83fd477 (DEFAULT_TABLE table=test-table [id=adc7c87081df4c50a3eba418fee58a0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:55:50.454480 14167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dbc11b9268945ca9f4e18f3d83fd477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:55:50.457934 14397 tablet_bootstrap.cc:492] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd: Bootstrap starting.
I20260810 14:55:50.458393 14398 tablet_bootstrap.cc:492] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c: Bootstrap starting.
I20260810 14:55:50.458693 14397 tablet_bootstrap.cc:654] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd: Neither blocks nor log segments found. Creating new log.
I20260810 14:55:50.459021 14398 tablet_bootstrap.cc:654] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c: Neither blocks nor log segments found. Creating new log.
I20260810 14:55:50.459311 14397 log.cc:824] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:50.459718 14398 log.cc:824] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:50.460215 14397 tablet_bootstrap.cc:492] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd: No bootstrap required, opened a new log
I20260810 14:55:50.460352 14397 ts_tablet_manager.cc:1397] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 14:55:50.460547 14398 tablet_bootstrap.cc:492] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c: No bootstrap required, opened a new log
I20260810 14:55:50.460603 14398 ts_tablet_manager.cc:1397] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:55:50.461939 14397 raft_consensus.cc:348] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:50.462122 14397 raft_consensus.cc:374] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:55:50.462037 14398 raft_consensus.cc:348] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:50.462174 14397 raft_consensus.cc:729] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f811229ee8704bc0bcfb7f059fd7a7dd, State: Initialized, Role: FOLLOWER
I20260810 14:55:50.462186 14398 raft_consensus.cc:374] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:55:50.462239 14398 raft_consensus.cc:729] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37350b06c8e2459ea7ca5cc6ead6807c, State: Initialized, Role: FOLLOWER
I20260810 14:55:50.462364 14398 consensus_queue.cc:260] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:50.462370 14397 consensus_queue.cc:260] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:50.462685 14238 heartbeater.cc:503] Master 127.30.74.126:45565 was elected leader, sending a full tablet report...
I20260810 14:55:50.462698 14397 ts_tablet_manager.cc:1428] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 14:55:50.462689 14398 ts_tablet_manager.cc:1428] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 14:55:50.462698 14386 heartbeater.cc:503] Master 127.30.74.126:45565 was elected leader, sending a full tablet report...
W20260810 14:55:50.521499 14239 tablet.cc:2369] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:55:50.618023 14086 tablet.cc:2369] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:55:50.663216 14387 tablet.cc:2369] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:55:50.677389 14401 raft_consensus.cc:482] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:55:50.677539 14401 raft_consensus.cc:504] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:50.677925 14401 leader_election.cc:288] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811), 35f5d65e079e47558d86a796107baddc (127.30.74.65:43907)
I20260810 14:55:50.682822 14404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.678264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:33585 (local address 127.30.74.65:43907)
0810 14:55:50.678423 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:50.678424 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:50.678432 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:55:50.678464 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:50.678465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:50.678495 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:50.678579 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:55:50.678885 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.679683 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.681592 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.682307 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.682386 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.682474 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:50.682662 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 14:55:50.682742 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260810 14:55:50.682821 14393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.678323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:40999 (local address 127.30.74.67:37811)
0810 14:55:50.678401 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:50.678403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:50.678442 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:55:50.678498 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:50.678499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:50.678508 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:50.678572 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:55:50.678883 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.679680 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.681595 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.682253 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.682358 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.682372 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:50.682662 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 14:55:50.682749 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 14:55:50.682778 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.678135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:43907 (local address 127.30.74.66:33585)
0810 14:55:50.678181 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:50.678194 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:50.678251 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:50.678656 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:50.678657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:50.678659 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:55:50.678757 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:50.678759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.679803 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.679804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:50.681412 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:50.681417 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.682425 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.682427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:50.682541 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.682543 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 14:55:50.682617 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 14:55:50.682669 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":195}
I20260810 14:55:50.682785 14403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.678161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:37811 (local address 127.30.74.66:40999)
0810 14:55:50.678379 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:50.678397 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:50.678450 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:50.678636 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:50.678637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:50.678640 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:55:50.678757 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:50.678759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.679802 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.679804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:50.681412 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:50.681417 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.682425 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.682427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:50.682537 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.682543 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:55:50.682617 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 14:55:50.682669 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":59,"thread_start_us":129,"threads_started":1}
I20260810 14:55:50.683172 14035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dbc11b9268945ca9f4e18f3d83fd477" candidate_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f5d65e079e47558d86a796107baddc" is_pre_election: true
I20260810 14:55:50.683342 14035 raft_consensus.cc:2454] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37350b06c8e2459ea7ca5cc6ead6807c in term 0.
I20260810 14:55:50.683279 14333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dbc11b9268945ca9f4e18f3d83fd477" candidate_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" is_pre_election: true
I20260810 14:55:50.683586 14333 raft_consensus.cc:2454] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37350b06c8e2459ea7ca5cc6ead6807c in term 0.
I20260810 14:55:50.683684 14114 leader_election.cc:302] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f5d65e079e47558d86a796107baddc, 37350b06c8e2459ea7ca5cc6ead6807c; no voters:
I20260810 14:55:50.683845 14401 raft_consensus.cc:2781] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:55:50.683917 14401 raft_consensus.cc:482] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:55:50.683936 14401 raft_consensus.cc:3037] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:55:50.684790 14401 raft_consensus.cc:504] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:50.684953 14401 leader_election.cc:288] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [CANDIDATE]: Term 1 election: Requested vote from peers f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811), 35f5d65e079e47558d86a796107baddc (127.30.74.65:43907)
I20260810 14:55:50.685197 14333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dbc11b9268945ca9f4e18f3d83fd477" candidate_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd"
I20260810 14:55:50.685211 14035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dbc11b9268945ca9f4e18f3d83fd477" candidate_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35f5d65e079e47558d86a796107baddc"
I20260810 14:55:50.685304 14035 raft_consensus.cc:3037] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:55:50.685312 14333 raft_consensus.cc:3037] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:55:50.686282 14035 raft_consensus.cc:2454] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37350b06c8e2459ea7ca5cc6ead6807c in term 1.
I20260810 14:55:50.686301 14333 raft_consensus.cc:2454] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37350b06c8e2459ea7ca5cc6ead6807c in term 1.
I20260810 14:55:50.686568 14114 leader_election.cc:302] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35f5d65e079e47558d86a796107baddc, 37350b06c8e2459ea7ca5cc6ead6807c; no voters:
I20260810 14:55:50.686713 14401 raft_consensus.cc:2781] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:55:50.686897 14401 raft_consensus.cc:686] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [term 1 LEADER]: Becoming Leader. State: Replica: 37350b06c8e2459ea7ca5cc6ead6807c, State: Running, Role: LEADER
I20260810 14:55:50.687005 14401 consensus_queue.cc:237] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:50.688043 13873 catalog_manager.cc:5495] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c reported cstate change: term changed from 0 to 1, leader changed from <none> to 37350b06c8e2459ea7ca5cc6ead6807c (127.30.74.66). New cstate: current_term: 1 leader_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } health_report { overall_health: UNKNOWN } } }
I20260810 14:55:50.737579 14394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.734979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36524 (local address 127.30.74.66:43511)
0810 14:55:50.735064 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:50.735067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:50.735074 (+ 7us) server_negotiation.cc:407] Connection header received
0810 14:55:50.735124 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:50.735125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:50.735144 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:50.735229 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:55:50.735354 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.735844 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.736687 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:50.736898 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:50.736992 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:50.737183 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:55:50.737303 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:55:50.737399 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:50.737423 (+ 24us) server_negotiation.cc:299] Negotiation successful
0810 14:55:50.737447 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 14:55:50.739663 14333 raft_consensus.cc:1261] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd [term 1 FOLLOWER]: Refusing update from remote peer 37350b06c8e2459ea7ca5cc6ead6807c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:55:50.739663 14035 raft_consensus.cc:1261] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Refusing update from remote peer 37350b06c8e2459ea7ca5cc6ead6807c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:55:50.740118 14405 consensus_queue.cc:1035] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [LEADER]: Connected to new peer: Peer: permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:55:50.740345 14401 consensus_queue.cc:1035] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:55:50.742969 14410 mvcc.cc:205] Tried to move back new op lower bound from 7316986883026378752 to 7316986882814697472. Current Snapshot: MvccSnapshot[applied={T|T < 7316986883026378752}]
I20260810 14:55:50.745362 13872 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:51674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:55:50.745496 13872 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:51674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:55:50.746447 13872 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6: Sending DeleteTablet for 3 replicas of tablet 5dbc11b9268945ca9f4e18f3d83fd477
I20260810 14:55:50.746912 14311 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dbc11b9268945ca9f4e18f3d83fd477 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:55:50 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:43714
I20260810 14:55:50.746953 14009 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dbc11b9268945ca9f4e18f3d83fd477 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:55:50 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:55170
I20260810 14:55:50.747017 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 13912
I20260810 14:55:50.749408 14167 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dbc11b9268945ca9f4e18f3d83fd477 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:55:50 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:39908
I20260810 14:55:50.749625 14420 tablet_replica.cc:330] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd: stopping tablet replica
I20260810 14:55:50.749677 14422 tablet_replica.cc:330] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c: stopping tablet replica
I20260810 14:55:50.749790 14422 raft_consensus.cc:2229] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:55:50.749790 14420 raft_consensus.cc:2229] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:55:50.749941 14422 raft_consensus.cc:2258] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:55:50.749956 14420 raft_consensus.cc:2258] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:55:50.750465 14422 ts_tablet_manager.cc:1905] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:55:50.750557 14420 ts_tablet_manager.cc:1905] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:55:50.752727 14422 ts_tablet_manager.cc:1918] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:55:50.752841 14422 log.cc:1196] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/5dbc11b9268945ca9f4e18f3d83fd477
I20260810 14:55:50.753140 14422 ts_tablet_manager.cc:1939] T 5dbc11b9268945ca9f4e18f3d83fd477 P 37350b06c8e2459ea7ca5cc6ead6807c: Deleting consensus metadata
I20260810 14:55:50.753608 14420 ts_tablet_manager.cc:1918] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:55:50.753695 14420 log.cc:1196] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/5dbc11b9268945ca9f4e18f3d83fd477
I20260810 14:55:50.753695 13853 catalog_manager.cc:4841] TS 37350b06c8e2459ea7ca5cc6ead6807c (127.30.74.66:43511): tablet 5dbc11b9268945ca9f4e18f3d83fd477 (table test-table [id=adc7c87081df4c50a3eba418fee58a0c]) successfully deleted
I20260810 14:55:50.753985 14420 ts_tablet_manager.cc:1939] T 5dbc11b9268945ca9f4e18f3d83fd477 P f811229ee8704bc0bcfb7f059fd7a7dd: Deleting consensus metadata
I20260810 14:55:50.754443 13855 catalog_manager.cc:4841] TS f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811): tablet 5dbc11b9268945ca9f4e18f3d83fd477 (table test-table [id=adc7c87081df4c50a3eba418fee58a0c]) successfully deleted
W20260810 14:55:50.757375 13853 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:43907 (error 108)
I20260810 14:55:50.757968 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:50.757828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:43907 (local address 127.0.0.1:48982)
0810 14:55:50.757897 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:50.757938 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:43907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 14:55:50.757980 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 14090
W20260810 14:55:50.760639 13853 catalog_manager.cc:4584] TS 35f5d65e079e47558d86a796107baddc (127.30.74.65:43907): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5dbc11b9268945ca9f4e18f3d83fd477: Network error: Client connection negotiation failed: client connection to 127.30.74.65:43907: connect: Connection refused (error 111)
I20260810 14:55:50.766659 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 14241
I20260810 14:55:50.775410 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 13833
I20260810 14:55:50.786123 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:45565
--webserver_interface=127.30.74.126
--webserver_port=39373
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:45565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:55:50.876896 14424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:50.877142 14424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:50.877167 14424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:50.878455 14424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:55:50.878515 14424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:50.878530 14424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:50.878544 14424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:55:50.878556 14424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:55:50.880234 14424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:45565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:45565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=39373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:50.880576 14424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:50.880862 14424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:50.886368 14430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:50.886402 14429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:50.886616 14432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:50.887149 14424 server_base.cc:1034] running on GCE node
I20260810 14:55:50.887554 14424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:50.887907 14424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:50.889109 14424 hybrid_clock.cc:648] HybridClock initialized: now 1786373750889022 us; error 107 us; skew 500 ppm
Aug 10 14:55:50 dist-test-slave-0b58 krb5kdc[13221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373750, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:50.891975 14424 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:55:50.892588 14424 webserver.cc:460] Webserver started at http://127.30.74.126:39373/ using document root <none> and password file <none>
I20260810 14:55:50.892859 14424 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:50.892935 14424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:50.894454 14424 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:55:50.895291 14440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:50.895496 14424 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:50.895614 14424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "49ae58c832d244d7aa6c2ef75d491ee6"
format_stamp: "Formatted at 2026-08-10 14:55:48 on dist-test-slave-0b58"
server_key: "697b38a47d3f3d802ed5d4eb5667d382"
server_key_iv: "a2f202b97c650beeef42a0c46a292dd3"
server_key_version: "encryptionkey@0"
I20260810 14:55:50.896027 14424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:50.921123 14424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:50.923712 14424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:50.923899 14424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:50.928705 14424 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:45565
I20260810 14:55:50.928766 14499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:45565 every 8 connection(s)
I20260810 14:55:50.929253 14424 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:55:50.931041 14500 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:55:50.931861 14500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6: Bootstrap starting.
I20260810 14:55:50.933471 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 14424
I20260810 14:55:50.933791 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:43907
--local_ip_for_outbound_sockets=127.30.74.65
--tserver_master_addrs=127.30.74.126:45565
--webserver_port=41527
--webserver_interface=127.30.74.65
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:55:50.940519 14500 log.cc:824] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:50.943051 14500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:55:50.943430 14500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6: Bootstrap complete.
I20260810 14:55:50.946024 14500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } }
I20260810 14:55:50.946354 14500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49ae58c832d244d7aa6c2ef75d491ee6, State: Initialized, Role: FOLLOWER
I20260810 14:55:50.946589 14500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } }
I20260810 14:55:50.946696 14500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:55:50.946755 14500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:55:50.946806 14500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 2 FOLLOWER]: Advancing to term 3
I20260810 14:55:50.947921 14500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } }
I20260810 14:55:50.948065 14500 leader_election.cc:302] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49ae58c832d244d7aa6c2ef75d491ee6; no voters:
I20260810 14:55:50.948309 14500 leader_election.cc:288] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 14:55:50.948364 14505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 14:55:50.948493 14505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [term 3 LEADER]: Becoming Leader. State: Replica: 49ae58c832d244d7aa6c2ef75d491ee6, State: Running, Role: LEADER
I20260810 14:55:50.948596 14505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } }
I20260810 14:55:50.948633 14500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:55:50.948910 14505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49ae58c832d244d7aa6c2ef75d491ee6. Latest consensus state: current_term: 3 leader_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } } }
I20260810 14:55:50.948987 14505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:55:50.949093 14506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49ae58c832d244d7aa6c2ef75d491ee6" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45565 } } }
I20260810 14:55:50.949203 14506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49ae58c832d244d7aa6c2ef75d491ee6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:55:50.949887 14510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:55:50.950870 14510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:55:50.951004 14510 catalog_manager.cc:1259] Loaded cluster ID: c9f5502c07d148af8c7c85b1b8ca927f
I20260810 14:55:50.951239 14510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 14:55:50.951491 14510 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f811634e9ad google::LogMessage::Fail() at ??:0
@ 0x7f811635188a google::LogMessage::SendToLog() at ??:0
@ 0x7f811634e4de google::LogMessage::Flush() at ??:0
@ 0x7f811634fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f811e2730f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f811e27c8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f811821c0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8118211954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f81190756db start_thread at ??:0
@ 0x7f8114e4c61f clone at ??:0
W20260810 14:55:51.051975 14503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:51.052238 14503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:51.052306 14503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:51.053583 14503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:51.053655 14503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:51.053704 14503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:55:51.055313 14503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:43907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=41527
--tserver_master_addrs=127.30.74.126:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:51.055629 14503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:51.055905 14503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:51.061391 14544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:51.061360 14542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:51.061383 14541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:51.061656 14503 server_base.cc:1034] running on GCE node
I20260810 14:55:51.062039 14503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:51.062374 14503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:51.063576 14503 hybrid_clock.cc:648] HybridClock initialized: now 1786373751063561 us; error 32 us; skew 500 ppm
Aug 10 14:55:51 dist-test-slave-0b58 krb5kdc[13221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373751, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:51.066905 14503 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:55:51.067512 14503 webserver.cc:460] Webserver started at http://127.30.74.65:41527/ using document root <none> and password file <none>
I20260810 14:55:51.067795 14503 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:51.067883 14503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:51.069583 14503 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:51.070463 14554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:51.070676 14503 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:51.070811 14503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "35f5d65e079e47558d86a796107baddc"
format_stamp: "Formatted at 2026-08-10 14:55:49 on dist-test-slave-0b58"
server_key: "7c36203ca4564283d00cd1fbf6936fdf"
server_key_iv: "efd880da0c141979b0a773c7b41a9508"
server_key_version: "encryptionkey@0"
I20260810 14:55:51.071220 14503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:51.082077 14503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:51.084760 14503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:51.084949 14503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:51.085239 14503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:55:51.085757 14561 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 14:55:51.087878 14503 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 14:55:51.087973 14503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 14:55:51.088027 14503 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 14:55:51.088811 14503 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 14:55:51.088884 14503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 14:55:51.088979 14561 tablet_bootstrap.cc:492] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc: Bootstrap starting.
I20260810 14:55:51.096517 14503 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:43907
I20260810 14:55:51.096692 14686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:43907 every 8 connection(s)
I20260810 14:55:51.096992 14503 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:55:51.100701 14561 log.cc:824] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc: Log is configured to *not* fsync() on all Append() calls
I20260810 14:55:51.100741 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 14503
I20260810 14:55:51.101094 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:43511
--local_ip_for_outbound_sockets=127.30.74.66
--tserver_master_addrs=127.30.74.126:45565
--webserver_port=39925
--webserver_interface=127.30.74.66
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:55:51.104070 14561 tablet_bootstrap.cc:492] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:55:51.104902 14561 tablet_bootstrap.cc:492] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc: Bootstrap complete.
I20260810 14:55:51.106356 14561 ts_tablet_manager.cc:1397] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.005s
I20260810 14:55:51.108244 14561 raft_consensus.cc:348] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:51.108738 14561 raft_consensus.cc:729] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f5d65e079e47558d86a796107baddc, State: Initialized, Role: FOLLOWER
I20260810 14:55:51.108955 14561 consensus_queue.cc:260] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:51.109328 14561 ts_tablet_manager.cc:1428] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
W20260810 14:55:51.172973 14690 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:55:51.097811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:57081)
0810 14:55:51.098015 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:51.098063 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:51.098633 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:51.172700 (+ 74067us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":111,"threads_started":1}
W20260810 14:55:51.173377 14687 heartbeater.cc:644] Failed to heartbeat to 127.30.74.126:45565 (0 consecutive failures): Network error: Failed to ping master at 127.30.74.126:45565: Client connection negotiation failed: client connection to 127.30.74.126:45565: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 14:55:51.173871 14690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:51.173696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:54815)
0810 14:55:51.173765 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:51.173834 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 14:55:51.174523 14690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:51.174404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:60471)
0810 14:55:51.174456 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:51.174492 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 14:55:51.174779 14687 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:55:51.195920 14692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:51.196164 14692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:51.196192 14692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:51.197603 14692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:51.197667 14692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:51.197700 14692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:55:51.199358 14692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:43511
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=39925
--tserver_master_addrs=127.30.74.126:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:51.199651 14692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:51.199914 14692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:51.205524 14702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:51.205524 14700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:51.205538 14699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:51.205685 14692 server_base.cc:1034] running on GCE node
I20260810 14:55:51.206029 14692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:51.206385 14692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:51.207574 14692 hybrid_clock.cc:648] HybridClock initialized: now 1786373751207553 us; error 27 us; skew 500 ppm
Aug 10 14:55:51 dist-test-slave-0b58 krb5kdc[13221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373751, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:51.210255 14692 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:55:51.210733 14692 webserver.cc:460] Webserver started at http://127.30.74.66:39925/ using document root <none> and password file <none>
I20260810 14:55:51.211000 14692 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:51.211074 14692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:51.212589 14692 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:55:51.213297 14710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:51.213482 14692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:55:51.213579 14692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "37350b06c8e2459ea7ca5cc6ead6807c"
format_stamp: "Formatted at 2026-08-10 14:55:49 on dist-test-slave-0b58"
server_key: "f1d92d3595555febf61bad4cbcdba331"
server_key_iv: "3d675aea46a24c953f8e5dfcdd5c13e3"
server_key_version: "encryptionkey@0"
I20260810 14:55:51.214120 14692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:51.232846 14692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:51.235040 14692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:51.235244 14692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:51.235541 14692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:55:51.235883 14692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:55:51.235939 14692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:51.235975 14692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:55:51.236003 14692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:51.242820 14692 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:43511
I20260810 14:55:51.242867 14841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:43511 every 8 connection(s)
I20260810 14:55:51.243290 14692 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:55:51.244670 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:51.244066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:50193)
0810 14:55:51.244406 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:51.244510 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":82,"threads_started":1}
W20260810 14:55:51.245174 14842 heartbeater.cc:644] Failed to heartbeat to 127.30.74.126:45565 (0 consecutive failures): Network error: Failed to ping master at 127.30.74.126:45565: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
I20260810 14:55:51.245612 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:51.245440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:52841)
0810 14:55:51.245548 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:51.245587 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 14:55:51.246086 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:51.245988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:37337)
0810 14:55:51.246035 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:51.246064 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 14:55:51.246256 14842 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 14:55:51.249399 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 14692
I20260810 14:55:51.249660 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:37811
--local_ip_for_outbound_sockets=127.30.74.67
--tserver_master_addrs=127.30.74.126:45565
--webserver_port=32985
--webserver_interface=127.30.74.67
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:55:51.338974 14845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:55:51.339223 14845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:55:51.339249 14845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:55:51.340512 14845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:55:51.340562 14845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:55:51.340592 14845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:55:51.342149 14845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:37811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=32985
--tserver_master_addrs=127.30.74.126:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:55:51.342377 14845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:55:51.342583 14845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:55:51.348579 14850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:51.348563 14851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:55:51.348546 14854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:55:51.349080 14845 server_base.cc:1034] running on GCE node
I20260810 14:55:51.349457 14845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:55:51.349803 14845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:55:51.350998 14845 hybrid_clock.cc:648] HybridClock initialized: now 1786373751350990 us; error 45 us; skew 500 ppm
Aug 10 14:55:51 dist-test-slave-0b58 krb5kdc[13221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373751, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:55:51.354120 14845 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:55:51.354688 14845 webserver.cc:460] Webserver started at http://127.30.74.67:32985/ using document root <none> and password file <none>
I20260810 14:55:51.354938 14845 fs_manager.cc:359] Metadata directory not provided
I20260810 14:55:51.355012 14845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:55:51.356634 14845 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 14:55:51.357381 14862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:51.357667 14845 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:55:51.357817 14845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "f811229ee8704bc0bcfb7f059fd7a7dd"
format_stamp: "Formatted at 2026-08-10 14:55:49 on dist-test-slave-0b58"
server_key: "55aa4cf31d46256f8ae4d9514e759fb0"
server_key_iv: "548790f074319408f5ad27339f1caef1"
server_key_version: "encryptionkey@0"
I20260810 14:55:51.358266 14845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:55:51.368983 14845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:55:51.371374 14845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:55:51.371582 14845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:55:51.371870 14845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:55:51.372232 14845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:55:51.372320 14845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:51.372386 14845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:55:51.372416 14845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:55:51.379505 14994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:37811 every 8 connection(s)
I20260810 14:55:51.379531 14845 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:37811
I20260810 14:55:51.380163 14845 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:55:51.381716 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:51.381119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:37511)
0810 14:55:51.381347 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:51.381519 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":84,"threads_started":1}
W20260810 14:55:51.382309 14995 heartbeater.cc:644] Failed to heartbeat to 127.30.74.126:45565 (0 consecutive failures): Network error: Failed to ping master at 127.30.74.126:45565: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
I20260810 14:55:51.382700 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:51.382591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:59587)
0810 14:55:51.382643 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:51.382676 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 14:55:51.383191 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:51.383063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:52239)
0810 14:55:51.383133 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:51.383166 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 14:55:51.383423 14995 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 14:55:51.386592 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 14845
I20260810 14:55:52.175625 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:52.175311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:51675)
0810 14:55:52.175479 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:52.175534 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":79,"threads_started":1}
I20260810 14:55:52.246927 15000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:52.246657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:59375)
0810 14:55:52.246817 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:52.246862 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
I20260810 14:55:52.384482 15001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:52.384012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:36095)
0810 14:55:52.384372 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:52.384410 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":241,"threads_started":1}
I20260810 14:55:52.585582 15002 raft_consensus.cc:482] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:55:52.585740 15002 raft_consensus.cc:504] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:52.586258 15002 leader_election.cc:288] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811), 37350b06c8e2459ea7ca5cc6ead6807c (127.30.74.66:43511)
Aug 10 14:55:52 dist-test-slave-0b58 krb5kdc[13221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373751, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.67@KRBTEST.COM
Aug 10 14:55:52 dist-test-slave-0b58 krb5kdc[13221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373751, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.66@KRBTEST.COM
I20260810 14:55:52.604900 15003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:52.586457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:43511 (local address 127.30.74.65:35323)
0810 14:55:52.586711 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:52.586721 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:52.586767 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:52.587721 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:52.587724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:52.588354 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:52.588499 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:52.588506 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:52.589710 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:52.589711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:52.590604 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:52.590608 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:52.591298 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:52.591299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:52.591435 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:52.597895 (+ 6460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:52.597909 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:52.600671 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:52.602933 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:52.602934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:52.602935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:52.603365 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:52.603677 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:52.603677 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:52.603678 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:52.603807 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:52.604310 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:52.604311 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:52.604448 (+ 137us) client_negotiation.cc:769] Sending connection context
0810 14:55:52.604567 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 14:55:52.604793 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":190,"thread_start_us":87,"threads_started":1}
I20260810 14:55:52.604910 15005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:52.586583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:34833 (local address 127.30.74.67:37811)
0810 14:55:52.586789 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:52.586798 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:52.586825 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:55:52.587674 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:52.587677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:52.587724 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:52.587842 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:52.588652 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:52.589569 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:52.590710 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:52.591164 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:52.591308 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:52.600380 (+ 9072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:52.600388 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:52.600388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:52.600408 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:52.602680 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:52.603452 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:52.603457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:52.603457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:52.603583 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:52.603891 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:52.603893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:52.603893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:52.604174 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:52.604265 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:52.604640 (+ 375us) server_negotiation.cc:299] Negotiation successful
0810 14:55:52.604780 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":84,"threads_started":1}
I20260810 14:55:52.604923 15006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:52.586612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:35323 (local address 127.30.74.66:43511)
0810 14:55:52.586753 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 14:55:52.586768 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:55:52.586811 (+ 43us) server_negotiation.cc:407] Connection header received
0810 14:55:52.587402 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:55:52.587406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:55:52.587470 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:55:52.587655 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:55:52.588662 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:52.589568 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:52.590719 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:55:52.591163 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:55:52.591312 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:52.600854 (+ 9542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:55:52.600859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:55:52.600859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:55:52.600872 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:55:52.602827 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:52.603483 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:52.603486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:52.603486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:52.603588 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:55:52.603889 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:55:52.603891 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:55:52.603895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:55:52.604176 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:55:52.604322 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 14:55:52.604626 (+ 304us) server_negotiation.cc:299] Negotiation successful
0810 14:55:52.604779 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
I20260810 14:55:52.605199 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:52.586452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:37811 (local address 127.30.74.65:34833)
0810 14:55:52.586516 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:52.586530 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:55:52.586602 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:55:52.587834 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:55:52.587834 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:55:52.588227 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:55:52.588498 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:52.588506 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:52.589700 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:52.589701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:52.590603 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:55:52.590606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:55:52.591294 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:55:52.591295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:55:52.591435 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:55:52.597903 (+ 6468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:55:52.597909 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:55:52.600058 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:55:52.602829 (+ 2771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:52.602843 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:52.602846 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:52.603349 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:52.603666 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:55:52.603667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:55:52.603667 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:55:52.603804 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:55:52.604301 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:55:52.604308 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:55:52.604449 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 14:55:52.604567 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 14:55:52.604881 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":171}
W20260810 14:55:52.606120 14557 leader_election.cc:341] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811): Not found: Tablet not found: 5dbc11b9268945ca9f4e18f3d83fd477
W20260810 14:55:52.606253 14555 leader_election.cc:341] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 37350b06c8e2459ea7ca5cc6ead6807c (127.30.74.66:43511): Not found: Tablet not found: 5dbc11b9268945ca9f4e18f3d83fd477
I20260810 14:55:52.606305 14555 leader_election.cc:302] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35f5d65e079e47558d86a796107baddc; no voters: 37350b06c8e2459ea7ca5cc6ead6807c, f811229ee8704bc0bcfb7f059fd7a7dd
I20260810 14:55:52.606459 15002 raft_consensus.cc:2726] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:55:53.176646 15007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:53.176334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:35589)
0810 14:55:53.176494 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:53.176584 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":67,"threads_started":1}
I20260810 14:55:53.247817 15008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:53.247551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:43435)
0810 14:55:53.247718 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:53.247768 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":69,"threads_started":1}
I20260810 14:55:53.385617 15009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:53.385321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:53207)
0810 14:55:53.385514 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:53.385565 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":73,"threads_started":1}
I20260810 14:55:54.177428 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:54.177148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:35271)
0810 14:55:54.177317 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:54.177372 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":70,"threads_started":1}
I20260810 14:55:54.248788 15011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:54.248520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:59053)
0810 14:55:54.248691 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:54.248738 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":117,"threads_started":1}
I20260810 14:55:54.386662 15012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:54.386348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:47395)
0810 14:55:54.386530 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:54.386610 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":79,"threads_started":1}
I20260810 14:55:54.460916 15013 raft_consensus.cc:482] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:55:54.461082 15013 raft_consensus.cc:504] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:54.461318 15013 leader_election.cc:288] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811), 37350b06c8e2459ea7ca5cc6ead6807c (127.30.74.66:43511)
W20260810 14:55:54.461800 14555 leader_election.cc:341] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 37350b06c8e2459ea7ca5cc6ead6807c (127.30.74.66:43511): Not found: Tablet not found: 5dbc11b9268945ca9f4e18f3d83fd477
W20260810 14:55:54.461891 14557 leader_election.cc:341] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811): Not found: Tablet not found: 5dbc11b9268945ca9f4e18f3d83fd477
I20260810 14:55:54.462002 14557 leader_election.cc:302] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35f5d65e079e47558d86a796107baddc; no voters: 37350b06c8e2459ea7ca5cc6ead6807c, f811229ee8704bc0bcfb7f059fd7a7dd
I20260810 14:55:54.462101 15013 raft_consensus.cc:2726] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:55:55.178416 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:55.178116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:38023)
0810 14:55:55.178281 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:55.178363 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":78,"threads_started":1}
I20260810 14:55:55.249670 15015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:55.249396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:58903)
0810 14:55:55.249562 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:55.249612 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":69,"threads_started":1}
I20260810 14:55:55.387684 15016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:55.387379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:46101)
0810 14:55:55.387547 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:55.387597 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":71,"threads_started":1}
I20260810 14:55:56.179499 15018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:56.179216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:39127)
0810 14:55:56.179399 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:56.179448 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":75,"threads_started":1}
I20260810 14:55:56.250569 15019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:56.250277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:44303)
0810 14:55:56.250453 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:56.250517 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":75,"threads_started":1}
I20260810 14:55:56.388774 15020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:56.388480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:52211)
0810 14:55:56.388672 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:56.388724 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":129,"threads_started":1}
I20260810 14:55:57.180570 15021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:57.180279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:37857)
0810 14:55:57.180465 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:57.180517 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":94,"threads_started":1}
I20260810 14:55:57.251663 15022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:57.251330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:39467)
0810 14:55:57.251542 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:57.251608 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":96,"threads_started":1}
I20260810 14:55:57.259927 15023 raft_consensus.cc:482] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:55:57.260041 15023 raft_consensus.cc:504] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:55:57.260303 15023 leader_election.cc:288] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811), 37350b06c8e2459ea7ca5cc6ead6807c (127.30.74.66:43511)
W20260810 14:55:57.260848 14557 leader_election.cc:341] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811): Not found: Tablet not found: 5dbc11b9268945ca9f4e18f3d83fd477
W20260810 14:55:57.260949 14555 leader_election.cc:341] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 37350b06c8e2459ea7ca5cc6ead6807c (127.30.74.66:43511): Not found: Tablet not found: 5dbc11b9268945ca9f4e18f3d83fd477
I20260810 14:55:57.261070 14555 leader_election.cc:302] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35f5d65e079e47558d86a796107baddc; no voters: 37350b06c8e2459ea7ca5cc6ead6807c, f811229ee8704bc0bcfb7f059fd7a7dd
I20260810 14:55:57.261165 15023 raft_consensus.cc:2726] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:55:57.389747 15024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:57.389443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:43331)
0810 14:55:57.389633 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:57.389695 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":91,"threads_started":1}
I20260810 14:55:58.181689 15026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:58.181412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:51779)
0810 14:55:58.181583 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:58.181633 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":75,"threads_started":1}
I20260810 14:55:58.252597 15027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:58.252360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:45945)
0810 14:55:58.252507 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:58.252542 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260810 14:55:58.390743 15028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:58.390429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:40705)
0810 14:55:58.390626 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:58.390687 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":85,"threads_started":1}
I20260810 14:55:59.182819 15029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:59.182515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:33101)
0810 14:55:59.182710 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:59.182761 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":96,"threads_started":1}
I20260810 14:55:59.253296 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:59.253033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:54327)
0810 14:55:59.253200 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:59.253246 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":83,"threads_started":1}
I20260810 14:55:59.391582 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:55:59.391337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:56261)
0810 14:55:59.391470 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 14:55:59.391532 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260810 14:56:00.184064 15032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:00.183712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:54603)
0810 14:56:00.183951 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:00.184010 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":98,"threads_started":1}
I20260810 14:56:00.254333 15033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:00.254024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:44997)
0810 14:56:00.254214 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:00.254277 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":81,"threads_started":1}
I20260810 14:56:00.392728 15034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:00.392389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:46587)
0810 14:56:00.392619 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:00.392677 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":65,"threads_started":1}
I20260810 14:56:01.185073 15035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:01.184816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:42267)
0810 14:56:01.184975 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:01.185022 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260810 14:56:01.255362 15037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:01.255056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:41127)
0810 14:56:01.255261 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:01.255311 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":98,"threads_started":1}
I20260810 14:56:01.393613 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:01.393341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:54647)
0810 14:56:01.393508 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:01.393562 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":74,"threads_started":1}
I20260810 14:56:01.584558 15040 raft_consensus.cc:482] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:56:01.584702 15040 raft_consensus.cc:504] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:56:01.584914 15040 leader_election.cc:288] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811), 37350b06c8e2459ea7ca5cc6ead6807c (127.30.74.66:43511)
W20260810 14:56:01.585388 14557 leader_election.cc:341] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811): Not found: Tablet not found: 5dbc11b9268945ca9f4e18f3d83fd477
W20260810 14:56:01.585636 14555 leader_election.cc:341] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 37350b06c8e2459ea7ca5cc6ead6807c (127.30.74.66:43511): Not found: Tablet not found: 5dbc11b9268945ca9f4e18f3d83fd477
I20260810 14:56:01.585734 14555 leader_election.cc:302] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35f5d65e079e47558d86a796107baddc; no voters: 37350b06c8e2459ea7ca5cc6ead6807c, f811229ee8704bc0bcfb7f059fd7a7dd
I20260810 14:56:01.585855 15040 raft_consensus.cc:2726] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:56:02.186074 15041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:02.185746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:52449)
0810 14:56:02.185963 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:02.186015 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":94,"threads_started":1}
I20260810 14:56:02.256407 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:02.256027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:33587)
0810 14:56:02.256236 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:02.256344 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":105,"threads_started":1}
I20260810 14:56:02.394588 15043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:02.394291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:56005)
0810 14:56:02.394490 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:02.394538 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":87,"threads_started":1}
I20260810 14:56:03.187050 15044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:03.186783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:39301)
0810 14:56:03.186948 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:03.186999 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
I20260810 14:56:03.257650 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:03.257303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:40695)
0810 14:56:03.257489 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:03.257563 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":79,"threads_started":1}
I20260810 14:56:03.395464 15046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:03.395185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:47519)
0810 14:56:03.395372 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:03.395416 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":76,"threads_started":1}
I20260810 14:56:04.187987 15048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:04.187696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:46615)
0810 14:56:04.187878 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:04.187935 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":103,"threads_started":1}
I20260810 14:56:04.258658 15049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:04.258339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:42363)
0810 14:56:04.258555 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:04.258606 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":72,"threads_started":1}
I20260810 14:56:04.396507 15050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:04.396166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:55589)
0810 14:56:04.396405 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:04.396459 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":228,"threads_started":1}
I20260810 14:56:04.443415 15051 raft_consensus.cc:482] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:56:04.443531 15051 raft_consensus.cc:504] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f811229ee8704bc0bcfb7f059fd7a7dd" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 37811 } } peers { permanent_uuid: "37350b06c8e2459ea7ca5cc6ead6807c" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 43511 } } peers { permanent_uuid: "35f5d65e079e47558d86a796107baddc" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 43907 } }
I20260810 14:56:04.443722 15051 leader_election.cc:288] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811), 37350b06c8e2459ea7ca5cc6ead6807c (127.30.74.66:43511)
W20260810 14:56:04.444137 14557 leader_election.cc:341] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f811229ee8704bc0bcfb7f059fd7a7dd (127.30.74.67:37811): Not found: Tablet not found: 5dbc11b9268945ca9f4e18f3d83fd477
W20260810 14:56:04.444196 14555 leader_election.cc:341] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 37350b06c8e2459ea7ca5cc6ead6807c (127.30.74.66:43511): Not found: Tablet not found: 5dbc11b9268945ca9f4e18f3d83fd477
I20260810 14:56:04.444307 14555 leader_election.cc:302] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 35f5d65e079e47558d86a796107baddc; no voters: 37350b06c8e2459ea7ca5cc6ead6807c, f811229ee8704bc0bcfb7f059fd7a7dd
I20260810 14:56:04.444393 15051 raft_consensus.cc:2726] T 5dbc11b9268945ca9f4e18f3d83fd477 P 35f5d65e079e47558d86a796107baddc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:56:05.188853 15052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:05.188582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:52983)
0810 14:56:05.188756 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:05.188800 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":70,"threads_started":1}
I20260810 14:56:05.259778 15053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:05.259485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:47461)
0810 14:56:05.259674 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:05.259726 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":77,"threads_started":1}
I20260810 14:56:05.397341 15054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:05.397082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:38749)
0810 14:56:05.397232 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:05.397288 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
I20260810 14:56:06.189949 15055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:06.189636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.65:57949)
0810 14:56:06.189846 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:06.189898 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":70,"threads_started":1}
I20260810 14:56:06.260691 15056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:06.260357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.66:47053)
0810 14:56:06.260526 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:06.260610 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":66,"threads_started":1}
I20260810 14:56:06.387672 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 14503
I20260810 14:56:06.395370 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 14692
I20260810 14:56:06.400918 15057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:06.400644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45565 (local address 127.30.74.67:58569)
0810 14:56:06.400820 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:06.400869 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:45565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":73,"threads_started":1}
I20260810 14:56:06.405093 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 14845
2026-08-10T14:56:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18911 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:56:06 dist-test-slave-0b58 krb5kdc[15060](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:56:06 dist-test-slave-0b58 krb5kdc[15060](info): set up 2 sockets
Aug 10 14:56:06 dist-test-slave-0b58 krb5kdc[15060](info): commencing operation
krb5kdc: starting...
W20260810 14:56:07.552744 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:56:07 dist-test-slave-0b58 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:56:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:56:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:56:07.699970 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:39299
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:39299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:56:07.787178 15075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:56:07.787482 15075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:56:07.787541 15075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:56:07.788874 15075 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 14:56:07.788942 15075 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 14:56:07.788965 15075 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 14:56:07.788985 15075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:56:07.789017 15075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:56:07.789049 15075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:56:07.789079 15075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:56:07.789106 15075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:56:07.790732 15075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:39299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:39299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:56:07.791070 15075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:56:07.791342 15075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:56:07.797641 15081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:07.797662 15075 server_base.cc:1034] running on GCE node
W20260810 14:56:07.797641 15086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:07.797643 15083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:07.798585 15075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:56:07.798991 15075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:56:07.800170 15075 hybrid_clock.cc:648] HybridClock initialized: now 1786373767800150 us; error 39 us; skew 500 ppm
Aug 10 14:56:07 dist-test-slave-0b58 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373767, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:56:07.803470 15075 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:56:07.804014 15075 webserver.cc:460] Webserver started at http://127.30.74.126:40493/ using document root <none> and password file <none>
I20260810 14:56:07.804344 15075 fs_manager.cc:359] Metadata directory not provided
I20260810 14:56:07.804430 15075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:56:07.804601 15075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:56:07.805837 15075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "7cf71e9ce81340fd9b668d7233c75a59"
format_stamp: "Formatted at 2026-08-10 14:56:07 on dist-test-slave-0b58"
server_key: "1ef860246939163d763c88364019d165"
server_key_iv: "5c9339b1725aaa597c3fcbaaf34bdf9a"
server_key_version: "encryptionkey@0"
I20260810 14:56:07.806396 15075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "7cf71e9ce81340fd9b668d7233c75a59"
format_stamp: "Formatted at 2026-08-10 14:56:07 on dist-test-slave-0b58"
server_key: "1ef860246939163d763c88364019d165"
server_key_iv: "5c9339b1725aaa597c3fcbaaf34bdf9a"
server_key_version: "encryptionkey@0"
I20260810 14:56:07.808008 15075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:56:07.808976 15093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:07.809165 15075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:56:07.809290 15075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "7cf71e9ce81340fd9b668d7233c75a59"
format_stamp: "Formatted at 2026-08-10 14:56:07 on dist-test-slave-0b58"
server_key: "1ef860246939163d763c88364019d165"
server_key_iv: "5c9339b1725aaa597c3fcbaaf34bdf9a"
server_key_version: "encryptionkey@0"
I20260810 14:56:07.809381 15075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:56:07.825899 15075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:56:07.828244 15075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:56:07.828500 15075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:56:07.833012 15075 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:39299
I20260810 14:56:07.833051 15155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:39299 every 8 connection(s)
I20260810 14:56:07.833449 15075 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:56:07.834466 15156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:56:07.836531 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 15075
I20260810 14:56:07.836642 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:56:07.836882 31017 external_mini_cluster.cc:1428] Setting key 34d24a0e43133c175c16a21c6a33fb4f
I20260810 14:56:07.837267 15156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Bootstrap starting.
I20260810 14:56:07.838083 15156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Neither blocks nor log segments found. Creating new log.
I20260810 14:56:07.838441 15156 log.cc:824] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Log is configured to *not* fsync() on all Append() calls
I20260810 14:56:07.839316 15156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: No bootstrap required, opened a new log
I20260810 14:56:07.841766 15156 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } }
I20260810 14:56:07.842000 15156 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:56:07.842036 15156 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cf71e9ce81340fd9b668d7233c75a59, State: Initialized, Role: FOLLOWER
I20260810 14:56:07.842175 15156 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } }
I20260810 14:56:07.842254 15156 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:56:07.842286 15156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:56:07.842324 15156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:56:07.843145 15156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } }
I20260810 14:56:07.843286 15156 leader_election.cc:302] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cf71e9ce81340fd9b668d7233c75a59; no voters:
I20260810 14:56:07.843547 15156 leader_election.cc:288] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:56:07.843583 15162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:56:07.843724 15162 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 1 LEADER]: Becoming Leader. State: Replica: 7cf71e9ce81340fd9b668d7233c75a59, State: Running, Role: LEADER
I20260810 14:56:07.843875 15156 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:56:07.843891 15162 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } }
I20260810 14:56:07.844233 15162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cf71e9ce81340fd9b668d7233c75a59. Latest consensus state: current_term: 1 leader_uuid: "7cf71e9ce81340fd9b668d7233c75a59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } } }
I20260810 14:56:07.844273 15164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cf71e9ce81340fd9b668d7233c75a59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } } }
I20260810 14:56:07.844550 15164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [sys.catalog]: This master's current role is: LEADER
I20260810 14:56:07.844446 15162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [sys.catalog]: This master's current role is: LEADER
I20260810 14:56:07.844856 15172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 10 14:56:07 dist-test-slave-0b58 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:56:07.845490 15172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:56:07.847342 15172 catalog_manager.cc:1347] Generated new cluster ID: 5c97056981314348afe052dec8c4b9d9
I20260810 14:56:07.847396 15172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:56:07.855479 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:07.837468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60432 (local address 127.30.74.126:39299)
0810 14:56:07.837678 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:07.837689 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:07.837733 (+ 44us) server_negotiation.cc:407] Connection header received
0810 14:56:07.838357 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:07.838368 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:07.838438 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:07.838600 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:56:07.839429 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:07.840958 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:07.841843 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:07.842493 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:07.842606 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:07.846131 (+ 3525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:56:07.846135 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:56:07.846135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:56:07.846148 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:56:07.853437 (+ 7289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:07.854048 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:07.854052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:07.854052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:07.854194 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:07.854441 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:07.854442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:07.854443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:07.854777 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:56:07.854931 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:07.855074 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 14:56:07.855227 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260810 14:56:07.857883 15172 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:56:07.858536 15172 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:56:07.861971 15172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Generated new TSK 0
I20260810 14:56:07.862354 15172 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:56:07.896243 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39299
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:56:07.987016 15185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:56:07.987257 15185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:56:07.987293 15185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:56:07.988551 15185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:56:07.988600 15185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:56:07.988628 15185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:56:07.990188 15185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:56:07.990433 15185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:56:07.990644 15185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 14:56:07.996727 15185 server_base.cc:1034] running on GCE node
W20260810 14:56:07.996987 15194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:07.997220 15190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:07.997584 15191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:07.997890 15185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:56:07.998231 15185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:56:07.999423 15185 hybrid_clock.cc:648] HybridClock initialized: now 1786373767999410 us; error 32 us; skew 500 ppm
Aug 10 14:56:07 dist-test-slave-0b58 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373767, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:56:08.002451 15185 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:56:08.002990 15185 webserver.cc:460] Webserver started at http://127.30.74.65:34273/ using document root <none> and password file <none>
I20260810 14:56:08.003218 15185 fs_manager.cc:359] Metadata directory not provided
I20260810 14:56:08.003263 15185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:56:08.003368 15185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:56:08.004563 15185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d"
format_stamp: "Formatted at 2026-08-10 14:56:07 on dist-test-slave-0b58"
server_key: "277cef01196f4730b51d9d3ae3a21d6c"
server_key_iv: "6cea70d52a56a68cb0082ca632a68fe7"
server_key_version: "encryptionkey@0"
I20260810 14:56:08.004913 15185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d"
format_stamp: "Formatted at 2026-08-10 14:56:07 on dist-test-slave-0b58"
server_key: "277cef01196f4730b51d9d3ae3a21d6c"
server_key_iv: "6cea70d52a56a68cb0082ca632a68fe7"
server_key_version: "encryptionkey@0"
I20260810 14:56:08.006429 15185 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:56:08.007467 15201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:08.007740 15185 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:56:08.007869 15185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d"
format_stamp: "Formatted at 2026-08-10 14:56:07 on dist-test-slave-0b58"
server_key: "277cef01196f4730b51d9d3ae3a21d6c"
server_key_iv: "6cea70d52a56a68cb0082ca632a68fe7"
server_key_version: "encryptionkey@0"
I20260810 14:56:08.007946 15185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:56:08.018081 15185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:56:08.020776 15185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:56:08.020915 15185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:56:08.021122 15185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:56:08.021508 15185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:56:08.021551 15185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:08.021576 15185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:56:08.021588 15185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:08.028980 15185 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:37673
I20260810 14:56:08.029047 15345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:37673 every 8 connection(s)
I20260810 14:56:08.029502 15185 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:56:08.032866 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 15185
I20260810 14:56:08.033026 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:56:08.033272 31017 external_mini_cluster.cc:1428] Setting key 0d56c52b33456d1a9f37b710c9883746
Aug 10 14:56:08 dist-test-slave-0b58 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373767, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:56:08.044164 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.030654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:50883 (local address 127.30.74.126:39299)
0810 14:56:08.030753 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:08.030755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:08.030779 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:56:08.031416 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:08.031417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:08.031430 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:08.031501 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:56:08.033291 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.034174 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.035440 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.035785 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.035886 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.040117 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:56:08.040135 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:56:08.040136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:56:08.040147 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:56:08.041939 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:08.042680 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:08.042682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:08.042682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:08.042758 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:08.043180 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:08.043182 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:08.043182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:08.043328 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:56:08.043396 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:08.043977 (+ 581us) server_negotiation.cc:299] Negotiation successful
0810 14:56:08.044088 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:56:08.044356 15349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.030422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:50883)
0810 14:56:08.030636 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:08.030685 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:08.031246 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:08.031638 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:08.031643 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:08.032687 (+ 1044us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:56:08.033064 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:08.033075 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.034316 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.034318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:08.035301 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:08.035305 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.035988 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.035989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:08.036507 (+ 518us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.037350 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:56:08.037369 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:56:08.039942 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:56:08.042068 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:56:08.042083 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:56:08.042087 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:56:08.042528 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:56:08.042849 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:56:08.042850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:56:08.042850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:56:08.043043 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:56:08.043414 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:56:08.043424 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:56:08.043641 (+ 217us) client_negotiation.cc:769] Sending connection context
0810 14:56:08.043787 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 14:56:08.044002 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":123,"threads_started":1}
I20260810 14:56:08.045002 15346 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39299
I20260810 14:56:08.045127 15346 heartbeater.cc:464] Registering TS with master...
I20260810 14:56:08.045361 15346 heartbeater.cc:511] Master 127.30.74.126:39299 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
I20260810 14:56:08.046065 15114 ts_manager.cc:194] Registered new tserver with Master: b0b3ea338c174c8c95aa3aa912ba6c3d (127.30.74.65:37673)
I20260810 14:56:08.046701 15114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:50883
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:56:08.069479 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39299
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:56:08.160516 15352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:56:08.160773 15352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:56:08.160802 15352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:56:08.162072 15352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:56:08.162127 15352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:56:08.162158 15352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:56:08.163771 15352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:56:08.164247 15352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:56:08.164549 15352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:56:08.170830 15361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:08.170845 15358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:08.170840 15357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:08.171161 15352 server_base.cc:1034] running on GCE node
I20260810 14:56:08.171411 15352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:56:08.171746 15352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:56:08.172935 15352 hybrid_clock.cc:648] HybridClock initialized: now 1786373768172920 us; error 35 us; skew 500 ppm
Aug 10 14:56:08 dist-test-slave-0b58 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373768, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:56:08.176132 15352 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:56:08.176775 15352 webserver.cc:460] Webserver started at http://127.30.74.66:45401/ using document root <none> and password file <none>
I20260810 14:56:08.177044 15352 fs_manager.cc:359] Metadata directory not provided
I20260810 14:56:08.177096 15352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:56:08.177343 15352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:56:08.179173 15352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "f15189643c4542a6bea41b24510fe0c6"
format_stamp: "Formatted at 2026-08-10 14:56:08 on dist-test-slave-0b58"
server_key: "fb5cedba153838e03cdf33f71613e17a"
server_key_iv: "649d3e08427b376ab6b6e2d661091228"
server_key_version: "encryptionkey@0"
I20260810 14:56:08.179688 15352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "f15189643c4542a6bea41b24510fe0c6"
format_stamp: "Formatted at 2026-08-10 14:56:08 on dist-test-slave-0b58"
server_key: "fb5cedba153838e03cdf33f71613e17a"
server_key_iv: "649d3e08427b376ab6b6e2d661091228"
server_key_version: "encryptionkey@0"
I20260810 14:56:08.181489 15352 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260810 14:56:08.182933 15369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:08.183281 15352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:56:08.183425 15352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "f15189643c4542a6bea41b24510fe0c6"
format_stamp: "Formatted at 2026-08-10 14:56:08 on dist-test-slave-0b58"
server_key: "fb5cedba153838e03cdf33f71613e17a"
server_key_iv: "649d3e08427b376ab6b6e2d661091228"
server_key_version: "encryptionkey@0"
I20260810 14:56:08.183526 15352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:56:08.207275 15352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:56:08.210225 15352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:56:08.210454 15352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:56:08.210749 15352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:56:08.211140 15352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:56:08.211199 15352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:08.211244 15352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:56:08.211270 15352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:08.219015 15352 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:38435
I20260810 14:56:08.219058 15501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:38435 every 8 connection(s)
I20260810 14:56:08.219486 15352 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
Aug 10 14:56:08 dist-test-slave-0b58 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373768, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:56:08.227034 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 15352
I20260810 14:56:08.227164 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:56:08.227428 31017 external_mini_cluster.cc:1428] Setting key d176c7903f1212ca16f519dd3c39cb50
I20260810 14:56:08.232313 15505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.220410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:35837)
0810 14:56:08.220605 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:08.220648 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:08.221185 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:08.221465 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:08.221469 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:08.222096 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:56:08.222356 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:08.222362 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.223145 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.223146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:08.224028 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:08.224031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.224566 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.224568 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:08.224728 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.225372 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:56:08.225382 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:56:08.226991 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:56:08.229916 (+ 2925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:56:08.229929 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:56:08.229949 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:56:08.230377 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:56:08.230658 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:56:08.230659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:56:08.230659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:56:08.230880 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:56:08.231255 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:56:08.231265 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:56:08.231652 (+ 387us) client_negotiation.cc:769] Sending connection context
0810 14:56:08.231767 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 14:56:08.231993 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260810 14:56:08.233045 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.220399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:35837 (local address 127.30.74.126:39299)
0810 14:56:08.220489 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:08.220492 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:08.220840 (+ 348us) server_negotiation.cc:407] Connection header received
0810 14:56:08.221329 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:08.221331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:08.221343 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:08.221403 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:56:08.222484 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.223023 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.224146 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.224453 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.224537 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.227134 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:56:08.227137 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:56:08.227137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:56:08.227165 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:56:08.229799 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:08.230504 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:08.230505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:08.230506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:08.230596 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:08.230955 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:08.230956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:08.230956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:08.231144 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:56:08.232779 (+ 1635us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:08.232835 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 14:56:08.232950 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 14:56:08.233749 15502 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39299
I20260810 14:56:08.233927 15502 heartbeater.cc:464] Registering TS with master...
I20260810 14:56:08.234220 15502 heartbeater.cc:511] Master 127.30.74.126:39299 requested a full tablet report, sending...
I20260810 14:56:08.234824 15114 ts_manager.cc:194] Registered new tserver with Master: f15189643c4542a6bea41b24510fe0c6 (127.30.74.66:38435)
I20260810 14:56:08.235332 15114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:35837
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:56:08.269148 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39299
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:56:08.360977 15508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:56:08.361238 15508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:56:08.361268 15508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:56:08.362723 15508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:56:08.362782 15508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:56:08.362812 15508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:56:08.364516 15508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:56:08.364748 15508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:56:08.364964 15508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:56:08.370927 15515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:08.370891 15517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:08.370908 15514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:08.371212 15508 server_base.cc:1034] running on GCE node
I20260810 14:56:08.371538 15508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:56:08.371850 15508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:56:08.373056 15508 hybrid_clock.cc:648] HybridClock initialized: now 1786373768373042 us; error 32 us; skew 500 ppm
Aug 10 14:56:08 dist-test-slave-0b58 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373768, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:56:08.376055 15508 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:56:08.376617 15508 webserver.cc:460] Webserver started at http://127.30.74.67:45565/ using document root <none> and password file <none>
I20260810 14:56:08.376838 15508 fs_manager.cc:359] Metadata directory not provided
I20260810 14:56:08.376883 15508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:56:08.376987 15508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:56:08.378108 15508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "b36e5f7583b64495bcdf11bc656ce923"
format_stamp: "Formatted at 2026-08-10 14:56:08 on dist-test-slave-0b58"
server_key: "3ba740b356b172cf85845c4be1128584"
server_key_iv: "41868100cd521213b62210db9df26a52"
server_key_version: "encryptionkey@0"
I20260810 14:56:08.378436 15508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "b36e5f7583b64495bcdf11bc656ce923"
format_stamp: "Formatted at 2026-08-10 14:56:08 on dist-test-slave-0b58"
server_key: "3ba740b356b172cf85845c4be1128584"
server_key_iv: "41868100cd521213b62210db9df26a52"
server_key_version: "encryptionkey@0"
I20260810 14:56:08.379833 15508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:56:08.380686 15526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:08.380889 15508 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 14:56:08.380995 15508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "b36e5f7583b64495bcdf11bc656ce923"
format_stamp: "Formatted at 2026-08-10 14:56:08 on dist-test-slave-0b58"
server_key: "3ba740b356b172cf85845c4be1128584"
server_key_iv: "41868100cd521213b62210db9df26a52"
server_key_version: "encryptionkey@0"
I20260810 14:56:08.381055 15508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:56:08.402268 15508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:56:08.405123 15508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:56:08.405262 15508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:56:08.405484 15508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:56:08.405844 15508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:56:08.405886 15508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:08.405910 15508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:56:08.405922 15508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:08.412487 15508 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:35443
I20260810 14:56:08.412595 15665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:35443 every 8 connection(s)
I20260810 14:56:08.412977 15508 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:56:08.416028 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 15508
I20260810 14:56:08.416169 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:56:08.416451 31017 external_mini_cluster.cc:1428] Setting key 118d6a997c9b58e5afae7661cb38afae
Aug 10 14:56:08 dist-test-slave-0b58 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373768, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:56:08.424333 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.413857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:33377 (local address 127.30.74.126:39299)
0810 14:56:08.413907 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:08.413909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:08.414127 (+ 218us) server_negotiation.cc:407] Connection header received
0810 14:56:08.414633 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:08.414634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:08.414647 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:08.414703 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:56:08.415699 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.416343 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.417866 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.418072 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.418142 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.421140 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:56:08.421142 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:56:08.421143 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:56:08.421152 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:56:08.422685 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:08.423177 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:08.423178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:08.423178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:08.423254 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:08.423550 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:08.423551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:08.423552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:08.423696 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:56:08.423780 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:08.424102 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 14:56:08.424195 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:56:08.424571 15668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.413816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:33377)
0810 14:56:08.413991 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:08.414031 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:08.414536 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:08.414762 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:08.414765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:08.415393 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:56:08.415609 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:08.415614 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.416466 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.416467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:08.417745 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:08.417749 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.418168 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.418169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:08.418291 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.418919 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:56:08.418928 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:56:08.421008 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:56:08.422808 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:56:08.422818 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:56:08.422821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:56:08.423086 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:56:08.423343 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:56:08.423344 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:56:08.423344 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:56:08.423478 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:56:08.423769 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:56:08.423778 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:56:08.423950 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 14:56:08.424066 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 14:56:08.424294 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260810 14:56:08.425139 15666 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39299
I20260810 14:56:08.425239 15666 heartbeater.cc:464] Registering TS with master...
I20260810 14:56:08.425411 15666 heartbeater.cc:511] Master 127.30.74.126:39299 requested a full tablet report, sending...
I20260810 14:56:08.425796 15114 ts_manager.cc:194] Registered new tserver with Master: b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67:35443)
I20260810 14:56:08.426240 15114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:33377
I20260810 14:56:08.428401 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:56:08.436704 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.429942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60434 (local address 127.30.74.126:39299)
0810 14:56:08.430027 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:08.430029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:08.430121 (+ 92us) server_negotiation.cc:407] Connection header received
0810 14:56:08.430177 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:08.430177 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:08.430187 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:08.430236 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:56:08.430988 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.431530 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.432368 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.432581 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.432645 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.434077 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:56:08.434079 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:56:08.434079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:56:08.434087 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:56:08.435463 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:08.435835 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:08.435837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:08.435837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:08.435898 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:08.436080 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:08.436080 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:08.436081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:08.436235 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:56:08.436399 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:08.436505 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 14:56:08.436584 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 14:56:08.438892 15114 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:56:08.440673 15114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:56:08.448421 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.443505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:37673 (local address 127.0.0.1:57642)
0810 14:56:08.443732 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:08.443793 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:08.443904 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:08.444423 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:08.444426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:08.444428 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:56:08.444519 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:08.444521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.445895 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.445896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:08.446981 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:08.446984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.447913 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.447914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:08.448005 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.448024 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 14:56:08.448127 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 14:56:08.448197 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":45,"thread_start_us":94,"threads_started":1}
I20260810 14:56:08.448570 15678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.443759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:38435 (local address 127.0.0.1:41580)
0810 14:56:08.444062 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:08.444074 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:08.444160 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:08.444834 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:08.444835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:08.444837 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:56:08.444877 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:08.444879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.445873 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.445874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:08.447345 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:08.447348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.448331 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.448332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:08.448399 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.448405 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:56:08.448440 (+ 35us) client_negotiation.cc:240] Negotiation successful
0810 14:56:08.448492 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":115,"thread_start_us":88,"threads_started":1}
I20260810 14:56:08.448347 15684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.443766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57642 (local address 127.30.74.65:37673)
0810 14:56:08.444170 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:08.444175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:08.444186 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:56:08.444227 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:08.444230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:08.444247 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:08.444417 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:56:08.444583 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.445766 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.447096 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.447810 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.447891 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.447957 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:08.448148 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 14:56:08.448197 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":92,"threads_started":1}
I20260810 14:56:08.448819 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.443631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53632 (local address 127.30.74.67:35443)
0810 14:56:08.444356 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:08.444361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:08.444372 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:56:08.444412 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:08.444415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:08.444430 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:08.444518 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:56:08.444730 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.445898 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.447472 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.448074 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.448142 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.448202 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:08.448651 (+ 449us) server_negotiation.cc:299] Negotiation successful
0810 14:56:08.448720 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":115,"threads_started":1}
I20260810 14:56:08.448922 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.444024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41580 (local address 127.30.74.66:38435)
0810 14:56:08.444669 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:08.444673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:08.444681 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:56:08.444711 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:08.444713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:08.444724 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:08.444834 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:56:08.444971 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.445730 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.447445 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.448178 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.448679 (+ 501us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.448760 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:08.448787 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 14:56:08.448839 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":94,"threads_started":1}
I20260810 14:56:08.449330 15681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.443509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:35443 (local address 127.0.0.1:53632)
0810 14:56:08.443967 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:08.443978 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:08.444035 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:08.444531 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:08.444532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:08.444535 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:56:08.444595 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:08.444597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.446163 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.446163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:08.447345 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:08.447348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.448239 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.448240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:08.448419 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.448429 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:56:08.449207 (+ 778us) client_negotiation.cc:240] Negotiation successful
0810 14:56:08.449261 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":4,"thread_start_us":110,"threads_started":1}
I20260810 14:56:08.449673 15429 tablet_service.cc:1467] Processing CreateTablet for tablet e4a009a1bf994c7daba07ce2b426350b (DEFAULT_TABLE table=test-table [id=f051175713fc40e0a8f887366512b835]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:56:08.449671 15269 tablet_service.cc:1467] Processing CreateTablet for tablet e4a009a1bf994c7daba07ce2b426350b (DEFAULT_TABLE table=test-table [id=f051175713fc40e0a8f887366512b835]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:56:08.450088 15429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4a009a1bf994c7daba07ce2b426350b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:56:08.450088 15269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4a009a1bf994c7daba07ce2b426350b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:56:08.450212 15589 tablet_service.cc:1467] Processing CreateTablet for tablet e4a009a1bf994c7daba07ce2b426350b (DEFAULT_TABLE table=test-table [id=f051175713fc40e0a8f887366512b835]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:56:08.450627 15589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4a009a1bf994c7daba07ce2b426350b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:56:08.453892 15688 tablet_bootstrap.cc:492] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923: Bootstrap starting.
I20260810 14:56:08.454591 15688 tablet_bootstrap.cc:654] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923: Neither blocks nor log segments found. Creating new log.
I20260810 14:56:08.455199 15688 log.cc:824] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923: Log is configured to *not* fsync() on all Append() calls
I20260810 14:56:08.455631 15687 tablet_bootstrap.cc:492] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6: Bootstrap starting.
I20260810 14:56:08.456239 15688 tablet_bootstrap.cc:492] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923: No bootstrap required, opened a new log
I20260810 14:56:08.456358 15688 ts_tablet_manager.cc:1397] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260810 14:56:08.456451 15687 tablet_bootstrap.cc:654] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6: Neither blocks nor log segments found. Creating new log.
I20260810 14:56:08.457088 15687 log.cc:824] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6: Log is configured to *not* fsync() on all Append() calls
I20260810 14:56:08.457562 15686 tablet_bootstrap.cc:492] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d: Bootstrap starting.
I20260810 14:56:08.458271 15687 tablet_bootstrap.cc:492] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6: No bootstrap required, opened a new log
I20260810 14:56:08.458396 15687 ts_tablet_manager.cc:1397] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 14:56:08.458528 15686 tablet_bootstrap.cc:654] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d: Neither blocks nor log segments found. Creating new log.
I20260810 14:56:08.459007 15688 raft_consensus.cc:348] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } }
I20260810 14:56:08.459223 15686 log.cc:824] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:56:08.459254 15688 raft_consensus.cc:374] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:56:08.459672 15688 raft_consensus.cc:729] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b36e5f7583b64495bcdf11bc656ce923, State: Initialized, Role: FOLLOWER
I20260810 14:56:08.459880 15688 consensus_queue.cc:260] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } }
I20260810 14:56:08.460225 15688 ts_tablet_manager.cc:1428] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260810 14:56:08.460381 15666 heartbeater.cc:503] Master 127.30.74.126:39299 was elected leader, sending a full tablet report...
I20260810 14:56:08.460415 15686 tablet_bootstrap.cc:492] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d: No bootstrap required, opened a new log
I20260810 14:56:08.460484 15686 ts_tablet_manager.cc:1397] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 14:56:08.460601 15687 raft_consensus.cc:348] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } }
I20260810 14:56:08.460847 15687 raft_consensus.cc:374] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:56:08.460894 15687 raft_consensus.cc:729] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f15189643c4542a6bea41b24510fe0c6, State: Initialized, Role: FOLLOWER
I20260810 14:56:08.461019 15687 consensus_queue.cc:260] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } }
I20260810 14:56:08.461323 15502 heartbeater.cc:503] Master 127.30.74.126:39299 was elected leader, sending a full tablet report...
I20260810 14:56:08.461607 15687 ts_tablet_manager.cc:1428] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:56:08.463030 15686 raft_consensus.cc:348] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } }
I20260810 14:56:08.463197 15686 raft_consensus.cc:374] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:56:08.463222 15686 raft_consensus.cc:729] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b3ea338c174c8c95aa3aa912ba6c3d, State: Initialized, Role: FOLLOWER
I20260810 14:56:08.463351 15686 consensus_queue.cc:260] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } }
I20260810 14:56:08.463671 15686 ts_tablet_manager.cc:1428] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:56:08.463725 15346 heartbeater.cc:503] Master 127.30.74.126:39299 was elected leader, sending a full tablet report...
W20260810 14:56:08.470223 15504 tablet.cc:2369] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:56:08.491499 15692 raft_consensus.cc:482] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:56:08.491637 15692 raft_consensus.cc:504] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } }
I20260810 14:56:08.491981 15692 leader_election.cc:288] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0b3ea338c174c8c95aa3aa912ba6c3d (127.30.74.65:37673), f15189643c4542a6bea41b24510fe0c6 (127.30.74.66:38435)
I20260810 14:56:08.496528 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.492370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:60467 (local address 127.30.74.66:38435)
0810 14:56:08.492428 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:08.492431 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:08.492446 (+ 15us) server_negotiation.cc:407] Connection header received
0810 14:56:08.492519 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:08.492520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:08.492530 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:08.492622 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:56:08.492859 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.493714 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.495452 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.496074 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.496171 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.496181 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:08.496404 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 14:56:08.496452 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 14:56:08.496528 15684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.492364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:33147 (local address 127.30.74.65:37673)
0810 14:56:08.492412 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:08.492414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:08.492589 (+ 175us) server_negotiation.cc:407] Connection header received
0810 14:56:08.492624 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:08.492626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:08.492637 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:08.492700 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:56:08.492876 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.493742 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.495451 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.496074 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.496170 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.496181 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:08.496400 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 14:56:08.496449 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 14:56:08.496529 15668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.492215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:38435 (local address 127.30.74.67:60467)
0810 14:56:08.492356 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:08.492372 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:08.492440 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:08.492641 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:08.492642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:08.492644 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:56:08.492748 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:08.492751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.493846 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.493848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:08.495278 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:08.495283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.496183 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.496184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:08.496301 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.496303 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 14:56:08.496383 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 14:56:08.496420 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":112}
I20260810 14:56:08.496541 15695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.492282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:37673 (local address 127.30.74.67:33147)
0810 14:56:08.492492 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:08.492503 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:08.492566 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:08.492716 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:08.492718 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:08.492720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:56:08.492780 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:08.492781 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.493846 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.493848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:08.495278 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:08.495283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.496183 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.496184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:08.496296 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.496301 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:56:08.496364 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 14:56:08.496420 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":9,"thread_start_us":134,"threads_started":1}
I20260810 14:56:08.497045 15452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a009a1bf994c7daba07ce2b426350b" candidate_uuid: "b36e5f7583b64495bcdf11bc656ce923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f15189643c4542a6bea41b24510fe0c6" is_pre_election: true
I20260810 14:56:08.497251 15452 raft_consensus.cc:2454] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b36e5f7583b64495bcdf11bc656ce923 in term 0.
I20260810 14:56:08.497264 15292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a009a1bf994c7daba07ce2b426350b" candidate_uuid: "b36e5f7583b64495bcdf11bc656ce923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" is_pre_election: true
I20260810 14:56:08.497475 15292 raft_consensus.cc:2454] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b36e5f7583b64495bcdf11bc656ce923 in term 0.
I20260810 14:56:08.497555 15529 leader_election.cc:302] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b36e5f7583b64495bcdf11bc656ce923, f15189643c4542a6bea41b24510fe0c6; no voters:
I20260810 14:56:08.497727 15692 raft_consensus.cc:2781] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:56:08.497786 15692 raft_consensus.cc:482] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:56:08.497803 15692 raft_consensus.cc:3037] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:56:08.498693 15692 raft_consensus.cc:504] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } }
I20260810 14:56:08.498869 15692 leader_election.cc:288] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [CANDIDATE]: Term 1 election: Requested vote from peers b0b3ea338c174c8c95aa3aa912ba6c3d (127.30.74.65:37673), f15189643c4542a6bea41b24510fe0c6 (127.30.74.66:38435)
I20260810 14:56:08.499061 15292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a009a1bf994c7daba07ce2b426350b" candidate_uuid: "b36e5f7583b64495bcdf11bc656ce923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d"
I20260810 14:56:08.499073 15452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a009a1bf994c7daba07ce2b426350b" candidate_uuid: "b36e5f7583b64495bcdf11bc656ce923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f15189643c4542a6bea41b24510fe0c6"
I20260810 14:56:08.499166 15292 raft_consensus.cc:3037] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:56:08.499161 15452 raft_consensus.cc:3037] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:56:08.500025 15292 raft_consensus.cc:2454] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b36e5f7583b64495bcdf11bc656ce923 in term 1.
I20260810 14:56:08.500025 15452 raft_consensus.cc:2454] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b36e5f7583b64495bcdf11bc656ce923 in term 1.
I20260810 14:56:08.500321 15528 leader_election.cc:302] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0b3ea338c174c8c95aa3aa912ba6c3d, b36e5f7583b64495bcdf11bc656ce923; no voters:
I20260810 14:56:08.500448 15692 raft_consensus.cc:2781] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:56:08.500629 15692 raft_consensus.cc:686] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [term 1 LEADER]: Becoming Leader. State: Replica: b36e5f7583b64495bcdf11bc656ce923, State: Running, Role: LEADER
I20260810 14:56:08.500751 15692 consensus_queue.cc:237] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } }
I20260810 14:56:08.501820 15114 catalog_manager.cc:5495] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 reported cstate change: term changed from 0 to 1, leader changed from <none> to b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67). New cstate: current_term: 1 leader_uuid: "b36e5f7583b64495bcdf11bc656ce923" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } health_report { overall_health: UNKNOWN } } }
I20260810 14:56:08.523650 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.520866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.30.74.67:35443)
0810 14:56:08.520936 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:08.520939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:08.520950 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:56:08.520991 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:08.520992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:08.521013 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:08.521090 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:56:08.521298 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.521811 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.522655 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:08.522868 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:08.522939 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:08.523155 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:56:08.523306 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:56:08.523383 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:08.523529 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 14:56:08.523557 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 14:56:08.526244 15292 raft_consensus.cc:1261] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Refusing update from remote peer b36e5f7583b64495bcdf11bc656ce923: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:56:08.526409 15452 raft_consensus.cc:1261] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6 [term 1 FOLLOWER]: Refusing update from remote peer b36e5f7583b64495bcdf11bc656ce923: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:56:08.526729 15696 consensus_queue.cc:1035] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:56:08.526893 15692 consensus_queue.cc:1035] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:56:08.533082 15114 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:56:08.533226 15114 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:56:08.534363 15114 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Sending DeleteTablet for 3 replicas of tablet e4a009a1bf994c7daba07ce2b426350b
I20260810 14:56:08.534809 15269 tablet_service.cc:1514] Processing DeleteTablet for tablet e4a009a1bf994c7daba07ce2b426350b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:56:08 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:57642
I20260810 14:56:08.534853 15589 tablet_service.cc:1514] Processing DeleteTablet for tablet e4a009a1bf994c7daba07ce2b426350b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:56:08 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:53632
I20260810 14:56:08.534956 15429 tablet_service.cc:1514] Processing DeleteTablet for tablet e4a009a1bf994c7daba07ce2b426350b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:56:08 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:41580
I20260810 14:56:08.535156 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 15185
I20260810 14:56:08.535328 15710 tablet_replica.cc:330] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6: stopping tablet replica
I20260810 14:56:08.535449 15710 raft_consensus.cc:2229] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:56:08.535557 15710 raft_consensus.cc:2258] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:56:08.537333 15710 ts_tablet_manager.cc:1905] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:56:08.539775 15710 ts_tablet_manager.cc:1918] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:56:08.539891 15710 log.cc:1196] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/e4a009a1bf994c7daba07ce2b426350b
I20260810 14:56:08.540216 15710 ts_tablet_manager.cc:1939] T e4a009a1bf994c7daba07ce2b426350b P f15189643c4542a6bea41b24510fe0c6: Deleting consensus metadata
I20260810 14:56:08.540899 15096 catalog_manager.cc:4841] TS f15189643c4542a6bea41b24510fe0c6 (127.30.74.66:38435): tablet e4a009a1bf994c7daba07ce2b426350b (table test-table [id=f051175713fc40e0a8f887366512b835]) successfully deleted
I20260810 14:56:08.542901 15709 tablet_replica.cc:330] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923: stopping tablet replica
I20260810 14:56:08.543083 15709 raft_consensus.cc:2229] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:56:08.543282 15709 raft_consensus.cc:2258] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:56:08.543862 15709 ts_tablet_manager.cc:1905] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:56:08.546061 15709 ts_tablet_manager.cc:1918] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:56:08.546185 15709 log.cc:1196] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/e4a009a1bf994c7daba07ce2b426350b
I20260810 14:56:08.546509 15709 ts_tablet_manager.cc:1939] T e4a009a1bf994c7daba07ce2b426350b P b36e5f7583b64495bcdf11bc656ce923: Deleting consensus metadata
I20260810 14:56:08.547053 15096 catalog_manager.cc:4841] TS b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67:35443): tablet e4a009a1bf994c7daba07ce2b426350b (table test-table [id=f051175713fc40e0a8f887366512b835]) successfully deleted
W20260810 14:56:08.548712 15095 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:37673 (error 108)
I20260810 14:56:08.549322 15681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:08.549162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:37673 (local address 127.0.0.1:57652)
0810 14:56:08.549225 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:08.549270 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:37673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 14:56:08.549437 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 15352
W20260810 14:56:08.556557 15095 catalog_manager.cc:4584] TS b0b3ea338c174c8c95aa3aa912ba6c3d (127.30.74.65:37673): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4a009a1bf994c7daba07ce2b426350b: Network error: Client connection negotiation failed: client connection to 127.30.74.65:37673: connect: Connection refused (error 111)
I20260810 14:56:08.556620 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 15508
I20260810 14:56:08.564966 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 15075
I20260810 14:56:28.573148 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:39299
--webserver_interface=127.30.74.126
--webserver_port=40493
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:39299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:56:28.672087 15714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:56:28.672367 15714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:56:28.672394 15714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:56:28.673892 15714 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 14:56:28.673965 15714 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 14:56:28.673980 15714 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 14:56:28.673992 15714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:56:28.674006 15714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:56:28.674021 15714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:56:28.674032 15714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:56:28.674045 15714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:56:28.675823 15714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:39299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:39299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=40493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:56:28.676090 15714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:56:28.676340 15714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:56:28.689402 15723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:28.689424 15721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:28.689977 15720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:28.690110 15714 server_base.cc:1034] running on GCE node
I20260810 14:56:28.690498 15714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:56:28.691056 15714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:56:28.692348 15714 hybrid_clock.cc:648] HybridClock initialized: now 1786373788692310 us; error 62 us; skew 500 ppm
Aug 10 14:56:28 dist-test-slave-0b58 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373788, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:56:28.695768 15714 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:56:28.696477 15714 webserver.cc:460] Webserver started at http://127.30.74.126:40493/ using document root <none> and password file <none>
I20260810 14:56:28.696785 15714 fs_manager.cc:359] Metadata directory not provided
I20260810 14:56:28.696882 15714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:56:28.698493 15714 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:56:28.699347 15731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:28.699553 15714 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:56:28.699677 15714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "7cf71e9ce81340fd9b668d7233c75a59"
format_stamp: "Formatted at 2026-08-10 14:56:07 on dist-test-slave-0b58"
server_key: "1ef860246939163d763c88364019d165"
server_key_iv: "5c9339b1725aaa597c3fcbaaf34bdf9a"
server_key_version: "encryptionkey@0"
I20260810 14:56:28.700158 15714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:56:28.720801 15714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:56:28.723094 15714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:56:28.723323 15714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:56:28.728976 15714 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:39299
I20260810 14:56:28.729092 15790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:39299 every 8 connection(s)
I20260810 14:56:28.729596 15714 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:56:28.730726 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 15714
I20260810 14:56:28.731097 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:37673
--local_ip_for_outbound_sockets=127.30.74.65
--tserver_master_addrs=127.30.74.126:39299
--webserver_port=34273
--webserver_interface=127.30.74.65
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:56:28.731633 15791 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:56:28.732532 15791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Bootstrap starting.
I20260810 14:56:28.742877 15791 log.cc:824] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Log is configured to *not* fsync() on all Append() calls
I20260810 14:56:28.746892 15791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:56:28.747308 15791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Bootstrap complete.
I20260810 14:56:28.749963 15791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } }
I20260810 14:56:28.750278 15791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cf71e9ce81340fd9b668d7233c75a59, State: Initialized, Role: FOLLOWER
I20260810 14:56:28.750496 15791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } }
I20260810 14:56:28.750577 15791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:56:28.750638 15791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:56:28.750691 15791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:56:28.751828 15791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } }
I20260810 14:56:28.751927 15791 leader_election.cc:302] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cf71e9ce81340fd9b668d7233c75a59; no voters:
I20260810 14:56:28.752184 15791 leader_election.cc:288] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 14:56:28.752228 15795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 14:56:28.752395 15795 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 2 LEADER]: Becoming Leader. State: Replica: 7cf71e9ce81340fd9b668d7233c75a59, State: Running, Role: LEADER
I20260810 14:56:28.752496 15795 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } }
I20260810 14:56:28.752549 15791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:56:28.752780 15796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7cf71e9ce81340fd9b668d7233c75a59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } } }
I20260810 14:56:28.752879 15796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [sys.catalog]: This master's current role is: LEADER
I20260810 14:56:28.752777 15795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cf71e9ce81340fd9b668d7233c75a59. Latest consensus state: current_term: 2 leader_uuid: "7cf71e9ce81340fd9b668d7233c75a59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } } }
I20260810 14:56:28.753084 15795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [sys.catalog]: This master's current role is: LEADER
I20260810 14:56:28.753225 15800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:56:28.754228 15800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:56:28.754369 15800 catalog_manager.cc:1259] Loaded cluster ID: 5c97056981314348afe052dec8c4b9d9
I20260810 14:56:28.754668 15800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:56:28.755525 15800 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:56:28.755741 15800 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Loaded TSK: 0
I20260810 14:56:28.765683 15800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Generated new TSK 1
I20260810 14:56:28.765846 15800 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 14:56:28.842531 15793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:56:28.842836 15793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:56:28.842885 15793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:56:28.844321 15793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:56:28.844612 15793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:56:28.844703 15793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:56:28.846899 15793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:37673
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=34273
--tserver_master_addrs=127.30.74.126:39299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:56:28.847266 15793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:56:28.847540 15793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:56:28.853879 15822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:28.853900 15820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:28.853894 15819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:28.854519 15793 server_base.cc:1034] running on GCE node
I20260810 14:56:28.854805 15793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:56:28.855187 15793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:56:28.856369 15793 hybrid_clock.cc:648] HybridClock initialized: now 1786373788856343 us; error 32 us; skew 500 ppm
Aug 10 14:56:28 dist-test-slave-0b58 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373788, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:56:28.859411 15793 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:56:28.860417 15793 webserver.cc:460] Webserver started at http://127.30.74.65:34273/ using document root <none> and password file <none>
I20260810 14:56:28.861057 15793 fs_manager.cc:359] Metadata directory not provided
I20260810 14:56:28.861346 15793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:56:28.863669 15793 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:56:28.864508 15829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:28.864665 15793 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:56:28.864765 15793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d"
format_stamp: "Formatted at 2026-08-10 14:56:07 on dist-test-slave-0b58"
server_key: "277cef01196f4730b51d9d3ae3a21d6c"
server_key_iv: "6cea70d52a56a68cb0082ca632a68fe7"
server_key_version: "encryptionkey@0"
I20260810 14:56:28.865173 15793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:56:28.880878 15793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:56:28.883546 15793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:56:28.883682 15793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:56:28.884006 15793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:56:28.884614 15836 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 14:56:28.886258 15793 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 14:56:28.886324 15793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 14:56:28.886354 15793 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 14:56:28.887367 15793 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 14:56:28.887434 15793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 14:56:28.887468 15836 tablet_bootstrap.cc:492] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d: Bootstrap starting.
I20260810 14:56:28.895983 15793 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:37673
I20260810 14:56:28.896405 15956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:37673 every 8 connection(s)
I20260810 14:56:28.896560 15793 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:56:28.898699 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 15793
I20260810 14:56:28.899070 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:38435
--local_ip_for_outbound_sockets=127.30.74.66
--tserver_master_addrs=127.30.74.126:39299
--webserver_port=45401
--webserver_interface=127.30.74.66
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:56:28.900714 15836 log.cc:824] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:56:28.903963 15836 tablet_bootstrap.cc:492] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:56:28.904369 15836 tablet_bootstrap.cc:492] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d: Bootstrap complete.
I20260810 14:56:28.905635 15836 ts_tablet_manager.cc:1397] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.010s
I20260810 14:56:28.907333 15836 raft_consensus.cc:348] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } }
I20260810 14:56:28.907763 15836 raft_consensus.cc:729] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b3ea338c174c8c95aa3aa912ba6c3d, State: Initialized, Role: FOLLOWER
I20260810 14:56:28.907912 15836 consensus_queue.cc:260] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } }
I20260810 14:56:28.908281 15836 ts_tablet_manager.cc:1428] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
Aug 10 14:56:28 dist-test-slave-0b58 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373788, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:56:28.914278 15961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:28.897443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:39725 (local address 127.30.74.126:39299)
0810 14:56:28.900459 (+ 3016us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:28.900472 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:28.900527 (+ 55us) server_negotiation.cc:407] Connection header received
0810 14:56:28.900689 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:28.900697 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:28.900763 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:28.901021 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:56:28.902294 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:28.903792 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:28.905876 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:28.906552 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:28.906699 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:28.909946 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:56:28.909951 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:56:28.909951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:56:28.909973 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:56:28.911883 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:28.912482 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:28.912486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:28.912486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:28.912600 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:28.913139 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:28.913140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:28.913141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:28.913430 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:56:28.913551 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:28.913910 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 14:56:28.914052 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2940,"thread_start_us":92,"threads_started":1}
I20260810 14:56:28.914263 15959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:28.897346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:39725)
0810 14:56:28.897541 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:28.897587 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:28.898211 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:28.901198 (+ 2987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:28.901202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:28.901820 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:56:28.902098 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:28.902107 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:28.903951 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:28.903952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:28.905674 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:28.905678 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:28.906826 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:28.906828 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:28.906972 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:28.907659 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:56:28.907670 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:56:28.909780 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:56:28.912018 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:56:28.912028 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:56:28.912030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:56:28.912387 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:56:28.912688 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:56:28.912690 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:56:28.912690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:56:28.912831 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:56:28.913526 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:56:28.913534 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:56:28.913689 (+ 155us) client_negotiation.cc:769] Sending connection context
0810 14:56:28.913818 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 14:56:28.914039 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":110,"threads_started":1}
I20260810 14:56:28.915531 15957 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39299
I20260810 14:56:28.915668 15957 heartbeater.cc:464] Registering TS with master...
I20260810 14:56:28.915879 15957 heartbeater.cc:511] Master 127.30.74.126:39299 requested a full tablet report, sending...
I20260810 14:56:28.916726 15750 ts_manager.cc:194] Registered new tserver with Master: b0b3ea338c174c8c95aa3aa912ba6c3d (127.30.74.65:37673)
I20260810 14:56:28.917907 15750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:39725
I20260810 14:56:28.919234 15957 heartbeater.cc:503] Master 127.30.74.126:39299 was elected leader, sending a full tablet report...
I20260810 14:56:28.924314 15966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:28.918260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.30.74.65:37673)
0810 14:56:28.918442 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:28.918448 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:28.918827 (+ 379us) server_negotiation.cc:407] Connection header received
0810 14:56:28.918870 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:28.918872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:28.919024 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:28.919105 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:56:28.919608 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:28.920587 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:28.923084 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:28.923746 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:28.923871 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:28.923942 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:28.924131 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 14:56:28.924211 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":110,"threads_started":1}
I20260810 14:56:28.924343 15965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:28.917398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:37673 (local address 127.0.0.1:34774)
0810 14:56:28.918615 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:28.918657 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:28.918745 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:28.919174 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:28.919177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:28.919180 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:56:28.919280 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:28.919282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:28.920681 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:28.920683 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:28.922919 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:28.922924 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:28.923877 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:28.923879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:28.923983 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:28.923999 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 14:56:28.924108 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 14:56:28.924179 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1183,"thread_start_us":129,"threads_started":1}
I20260810 14:56:28.925557 15883 tablet_service.cc:1514] Processing DeleteTablet for tablet e4a009a1bf994c7daba07ce2b426350b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:56:08 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:34774
I20260810 14:56:28.926246 15967 tablet_replica.cc:330] stopping tablet replica
I20260810 14:56:28.926396 15967 raft_consensus.cc:2229] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:56:28.926487 15967 raft_consensus.cc:2258] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:56:28.926870 15967 ts_tablet_manager.cc:1905] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:56:28.930719 15967 ts_tablet_manager.cc:1918] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:56:28.930938 15967 log.cc:1196] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal/wals/e4a009a1bf994c7daba07ce2b426350b
I20260810 14:56:28.931195 15967 ts_tablet_manager.cc:1939] T e4a009a1bf994c7daba07ce2b426350b P b0b3ea338c174c8c95aa3aa912ba6c3d: Deleting consensus metadata
I20260810 14:56:28.931842 15734 catalog_manager.cc:4841] TS b0b3ea338c174c8c95aa3aa912ba6c3d (127.30.74.65:37673): tablet e4a009a1bf994c7daba07ce2b426350b (table test-table [id=f051175713fc40e0a8f887366512b835]) successfully deleted
W20260810 14:56:29.007671 15962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:56:29.007944 15962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:56:29.007983 15962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:56:29.009575 15962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:56:29.009634 15962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:56:29.009667 15962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:56:29.011811 15962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:38435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=45401
--tserver_master_addrs=127.30.74.126:39299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:56:29.012138 15962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:56:29.012455 15962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:56:29.019101 15974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:29.019368 15973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:29.019369 15976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:29.019744 15962 server_base.cc:1034] running on GCE node
I20260810 14:56:29.020044 15962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:56:29.020429 15962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:56:29.021680 15962 hybrid_clock.cc:648] HybridClock initialized: now 1786373789021658 us; error 36 us; skew 500 ppm
Aug 10 14:56:29 dist-test-slave-0b58 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373789, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:56:29.026371 15962 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:56:29.027072 15962 webserver.cc:460] Webserver started at http://127.30.74.66:45401/ using document root <none> and password file <none>
I20260810 14:56:29.027348 15962 fs_manager.cc:359] Metadata directory not provided
I20260810 14:56:29.027446 15962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:56:29.030277 15962 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260810 14:56:29.031433 15983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:29.031909 15962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:56:29.032063 15962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "f15189643c4542a6bea41b24510fe0c6"
format_stamp: "Formatted at 2026-08-10 14:56:08 on dist-test-slave-0b58"
server_key: "fb5cedba153838e03cdf33f71613e17a"
server_key_iv: "649d3e08427b376ab6b6e2d661091228"
server_key_version: "encryptionkey@0"
I20260810 14:56:29.032559 15962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:56:29.045941 15962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:56:29.049302 15962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:56:29.049489 15962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:56:29.049814 15962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:56:29.050171 15962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:56:29.050230 15962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:29.050274 15962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:56:29.050299 15962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:29.057452 15962 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:38435
I20260810 14:56:29.057492 16114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:38435 every 8 connection(s)
I20260810 14:56:29.057895 15962 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:56:29.058611 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 15962
I20260810 14:56:29.058974 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:35443
--local_ip_for_outbound_sockets=127.30.74.67
--tserver_master_addrs=127.30.74.126:39299
--webserver_port=45565
--webserver_interface=127.30.74.67
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:56:29 dist-test-slave-0b58 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373789, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:56:29.070662 15961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.058731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:39129 (local address 127.30.74.126:39299)
0810 14:56:29.058819 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:29.058822 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:29.059088 (+ 266us) server_negotiation.cc:407] Connection header received
0810 14:56:29.059686 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:29.059687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:29.059703 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:29.059763 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:56:29.061128 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.061700 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.063020 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.063274 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.063347 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:29.066347 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:56:29.066349 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:56:29.066350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:56:29.066358 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:56:29.068365 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:29.069054 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:29.069056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:29.069057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:29.069428 (+ 371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:29.069781 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:29.069783 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:29.069784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:29.069974 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:56:29.070061 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:29.070419 (+ 358us) server_negotiation.cc:299] Negotiation successful
0810 14:56:29.070542 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:56:29.070950 16117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.058774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:39129)
0810 14:56:29.058955 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:29.058995 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:29.059567 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:29.059849 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:29.059854 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:29.060746 (+ 892us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:56:29.060994 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:29.061001 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.061830 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.061831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:29.062899 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:29.062904 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.063383 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.063384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:29.063554 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:29.064311 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:56:29.064323 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:56:29.066187 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:56:29.068524 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:56:29.068535 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:56:29.068538 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:56:29.068953 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:56:29.069542 (+ 589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:56:29.069543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:56:29.069544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:56:29.069694 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:56:29.070051 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:56:29.070059 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:56:29.070238 (+ 179us) client_negotiation.cc:769] Sending connection context
0810 14:56:29.070392 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 14:56:29.070650 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":105,"threads_started":1}
I20260810 14:56:29.071451 16115 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39299
I20260810 14:56:29.071564 16115 heartbeater.cc:464] Registering TS with master...
I20260810 14:56:29.071823 16115 heartbeater.cc:511] Master 127.30.74.126:39299 requested a full tablet report, sending...
I20260810 14:56:29.072345 15750 ts_manager.cc:194] Registered new tserver with Master: f15189643c4542a6bea41b24510fe0c6 (127.30.74.66:38435)
I20260810 14:56:29.072793 15750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:39129
W20260810 14:56:29.154258 16118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:56:29.154568 16118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:56:29.154614 16118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:56:29.155900 16118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:56:29.155977 16118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:56:29.156029 16118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:56:29.157802 16118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:35443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=45565
--tserver_master_addrs=127.30.74.126:39299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:56:29.158113 16118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:56:29.158396 16118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:56:29.164103 16125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:29.164116 16129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:29.164180 16124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:29.164713 16118 server_base.cc:1034] running on GCE node
I20260810 14:56:29.165150 16118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:56:29.165490 16118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:56:29.166687 16118 hybrid_clock.cc:648] HybridClock initialized: now 1786373789166649 us; error 53 us; skew 500 ppm
Aug 10 14:56:29 dist-test-slave-0b58 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373789, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:56:29.169637 16118 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:56:29.170142 16118 webserver.cc:460] Webserver started at http://127.30.74.67:45565/ using document root <none> and password file <none>
I20260810 14:56:29.170359 16118 fs_manager.cc:359] Metadata directory not provided
I20260810 14:56:29.170418 16118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:56:29.171870 16118 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:56:29.172629 16136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:29.172892 16118 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:56:29.172999 16118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "b36e5f7583b64495bcdf11bc656ce923"
format_stamp: "Formatted at 2026-08-10 14:56:08 on dist-test-slave-0b58"
server_key: "3ba740b356b172cf85845c4be1128584"
server_key_iv: "41868100cd521213b62210db9df26a52"
server_key_version: "encryptionkey@0"
I20260810 14:56:29.173394 16118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:56:29.181742 16118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:56:29.184494 16118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:56:29.184649 16118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:56:29.184971 16118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:56:29.185331 16118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:56:29.185387 16118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:29.185429 16118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:56:29.185460 16118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:29.191931 16118 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:35443
I20260810 14:56:29.192068 16267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:35443 every 8 connection(s)
I20260810 14:56:29.192536 16118 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:56:29.197232 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 16118
Aug 10 14:56:29 dist-test-slave-0b58 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373789, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:56:29.203011 15961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.193357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:41241 (local address 127.30.74.126:39299)
0810 14:56:29.193457 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:29.193459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:29.193657 (+ 198us) server_negotiation.cc:407] Connection header received
0810 14:56:29.194141 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:29.194142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:29.194153 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:29.194198 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:56:29.195179 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.195678 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.196844 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.197104 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.197184 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:29.199640 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:56:29.199642 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:56:29.199642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:56:29.199650 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:56:29.201296 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:29.201771 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:29.201772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:29.201772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:29.201858 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:29.202192 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:29.202193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:29.202193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:29.202316 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:56:29.202374 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:29.202764 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 14:56:29.202886 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:56:29.203202 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.193335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:41241)
0810 14:56:29.193521 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:29.193560 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:29.194053 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:29.194264 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:29.194267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:29.194879 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:56:29.195085 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:29.195090 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.195776 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.195777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:29.196756 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:29.196758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.197215 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.197215 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:29.197321 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:29.197970 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:56:29.197982 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:56:29.199527 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:56:29.201403 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:56:29.201412 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:56:29.201415 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:56:29.201669 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:56:29.201940 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:56:29.201941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:56:29.201941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:56:29.202114 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:56:29.202393 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:56:29.202401 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:56:29.202595 (+ 194us) client_negotiation.cc:769] Sending connection context
0810 14:56:29.202732 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 14:56:29.202933 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":116,"threads_started":1}
I20260810 14:56:29.203886 16268 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39299
I20260810 14:56:29.204011 16268 heartbeater.cc:464] Registering TS with master...
I20260810 14:56:29.204293 16268 heartbeater.cc:511] Master 127.30.74.126:39299 requested a full tablet report, sending...
I20260810 14:56:29.204794 15750 ts_manager.cc:194] Registered new tserver with Master: b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67:35443)
I20260810 14:56:29.205022 16272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.197765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45458 (local address 127.30.74.126:39299)
0810 14:56:29.197930 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:29.197932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:29.197980 (+ 48us) server_negotiation.cc:407] Connection header received
0810 14:56:29.198039 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:29.198040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:29.198053 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:29.198127 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:56:29.198743 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.199353 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.200149 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.200434 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.200520 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:29.201622 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:56:29.201624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:56:29.201624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:56:29.201631 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:56:29.203464 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:29.203918 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:29.203919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:29.203919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:29.203998 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:29.204345 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:29.204347 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:29.204347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:29.204535 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:56:29.204767 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:29.204829 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 14:56:29.204927 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260810 14:56:29.205381 15750 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:41241
I20260810 14:56:29.205866 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:56:29.208854 16272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.206316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45472 (local address 127.30.74.126:39299)
0810 14:56:29.206363 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:29.206365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:29.206506 (+ 141us) server_negotiation.cc:407] Connection header received
0810 14:56:29.206551 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:29.206552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:29.206582 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:29.206648 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:56:29.206722 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.207223 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.208026 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.208346 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.208418 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:29.208610 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:56:29.208654 (+ 44us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 14:56:29.208803 (+ 149us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45472: token expired
Metrics: {"server-negotiator.queue_time_us":26}
W20260810 14:56:29.208935 16272 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45472: token expired
W20260810 14:56:29.209137 15671 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.74.126:39299)
I20260810 14:56:29.215401 16272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.209477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45476 (local address 127.30.74.126:39299)
0810 14:56:29.209518 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:29.209519 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:29.209555 (+ 36us) server_negotiation.cc:407] Connection header received
0810 14:56:29.209584 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:29.209584 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:29.209593 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:29.209634 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:56:29.210089 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.210571 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.211252 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.211434 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.211500 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:29.212733 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:56:29.212734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:56:29.212735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:56:29.212742 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:56:29.214066 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:29.214480 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:29.214481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:29.214482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:29.214544 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:29.214771 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:29.214772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:29.214772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:29.214930 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:56:29.215044 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:29.215207 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 14:56:29.215299 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 14:56:29.216799 15750 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:56:29.218451 15750 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:56:29.221020 15883 tablet_service.cc:1467] Processing CreateTablet for tablet 178b91bf8b9d4ec39c27aa5787f858dd (DEFAULT_TABLE table=test-table [id=f9830fc27d144f5aaf773ab8a830cd3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:56:29.221261 15883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 178b91bf8b9d4ec39c27aa5787f858dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:56:29.223067 15836 tablet_bootstrap.cc:492] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d: Bootstrap starting.
I20260810 14:56:29.223502 15836 tablet_bootstrap.cc:654] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d: Neither blocks nor log segments found. Creating new log.
I20260810 14:56:29.224154 15836 tablet_bootstrap.cc:492] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d: No bootstrap required, opened a new log
I20260810 14:56:29.224200 15836 ts_tablet_manager.cc:1397] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 14:56:29.224371 15836 raft_consensus.cc:348] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } }
I20260810 14:56:29.224478 15836 raft_consensus.cc:374] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:56:29.224527 15836 raft_consensus.cc:729] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b3ea338c174c8c95aa3aa912ba6c3d, State: Initialized, Role: FOLLOWER
I20260810 14:56:29.224576 15836 consensus_queue.cc:260] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } }
I20260810 14:56:29.224669 15836 ts_tablet_manager.cc:1428] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:29.225558 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.221092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39070 (local address 127.30.74.66:38435)
0810 14:56:29.221294 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:29.221301 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:29.221316 (+ 15us) server_negotiation.cc:407] Connection header received
0810 14:56:29.221352 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:29.221353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:29.221387 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:29.221479 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:56:29.221634 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.222686 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.224471 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.225230 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.225305 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:29.225375 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:29.225464 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 14:56:29.225497 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":130,"threads_started":1}
I20260810 14:56:29.225574 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.220863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:38435 (local address 127.0.0.1:39070)
0810 14:56:29.221008 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:29.221018 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:29.221063 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:29.221491 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:29.221492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:29.221495 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:56:29.221556 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:29.221558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.222783 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.222785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:29.224377 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:29.224381 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.225318 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.225319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:29.225382 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:29.225387 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:56:29.225452 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 14:56:29.225494 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260810 14:56:29.226619 15965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.220767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:35443 (local address 127.0.0.1:48264)
0810 14:56:29.220810 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:29.220822 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:29.220862 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:29.223329 (+ 2467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:29.223330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:29.223332 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:56:29.223374 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:29.223376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.224624 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.224624 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:29.225616 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:29.225618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.226447 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.226448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:29.226497 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:29.226500 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:56:29.226533 (+ 33us) client_negotiation.cc:240] Negotiation successful
0810 14:56:29.226569 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260810 14:56:29.226670 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.220871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48264 (local address 127.30.74.67:35443)
0810 14:56:29.223133 (+ 2262us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:29.223139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:29.223153 (+ 14us) server_negotiation.cc:407] Connection header received
0810 14:56:29.223199 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:29.223201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:29.223219 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:29.223317 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:56:29.223443 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.224524 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.225727 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.226372 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.226448 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:29.226511 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:29.226549 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 14:56:29.226596 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2214,"thread_start_us":88,"threads_started":1}
I20260810 14:56:29.226548 16039 tablet_service.cc:1467] Processing CreateTablet for tablet 178b91bf8b9d4ec39c27aa5787f858dd (DEFAULT_TABLE table=test-table [id=f9830fc27d144f5aaf773ab8a830cd3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:56:29.226917 16039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 178b91bf8b9d4ec39c27aa5787f858dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:56:29.227449 16193 tablet_service.cc:1467] Processing CreateTablet for tablet 178b91bf8b9d4ec39c27aa5787f858dd (DEFAULT_TABLE table=test-table [id=f9830fc27d144f5aaf773ab8a830cd3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:56:29.227762 16193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 178b91bf8b9d4ec39c27aa5787f858dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:56:29.230545 16278 tablet_bootstrap.cc:492] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6: Bootstrap starting.
I20260810 14:56:29.231021 16279 tablet_bootstrap.cc:492] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923: Bootstrap starting.
I20260810 14:56:29.231336 16278 tablet_bootstrap.cc:654] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6: Neither blocks nor log segments found. Creating new log.
I20260810 14:56:29.231746 16279 tablet_bootstrap.cc:654] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923: Neither blocks nor log segments found. Creating new log.
I20260810 14:56:29.232023 16278 log.cc:824] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6: Log is configured to *not* fsync() on all Append() calls
I20260810 14:56:29.232398 16279 log.cc:824] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923: Log is configured to *not* fsync() on all Append() calls
I20260810 14:56:29.233002 16278 tablet_bootstrap.cc:492] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6: No bootstrap required, opened a new log
I20260810 14:56:29.233083 16278 ts_tablet_manager.cc:1397] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 14:56:29.234510 16278 raft_consensus.cc:348] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } }
I20260810 14:56:29.234671 16278 raft_consensus.cc:374] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:56:29.234694 16278 raft_consensus.cc:729] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f15189643c4542a6bea41b24510fe0c6, State: Initialized, Role: FOLLOWER
I20260810 14:56:29.234831 16278 consensus_queue.cc:260] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } }
I20260810 14:56:29.235139 16278 ts_tablet_manager.cc:1428] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 14:56:29.235200 16115 heartbeater.cc:503] Master 127.30.74.126:39299 was elected leader, sending a full tablet report...
I20260810 14:56:29.235937 16279 tablet_bootstrap.cc:492] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923: No bootstrap required, opened a new log
I20260810 14:56:29.236022 16279 ts_tablet_manager.cc:1397] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 14:56:29.237912 16279 raft_consensus.cc:348] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } }
I20260810 14:56:29.238077 16279 raft_consensus.cc:374] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:56:29.238101 16279 raft_consensus.cc:729] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b36e5f7583b64495bcdf11bc656ce923, State: Initialized, Role: FOLLOWER
I20260810 14:56:29.238240 16279 consensus_queue.cc:260] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } }
I20260810 14:56:29.238561 16279 ts_tablet_manager.cc:1428] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 14:56:29.238591 16268 heartbeater.cc:503] Master 127.30.74.126:39299 was elected leader, sending a full tablet report...
W20260810 14:56:29.308492 16116 tablet.cc:2369] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:56:29.396754 16282 raft_consensus.cc:482] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:56:29.396900 16282 raft_consensus.cc:504] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } }
I20260810 14:56:29.397354 16282 leader_election.cc:288] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67:35443), b0b3ea338c174c8c95aa3aa912ba6c3d (127.30.74.65:37673)
I20260810 14:56:29.400518 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.397649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:50145 (local address 127.30.74.67:35443)
0810 14:56:29.397698 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:29.397700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:29.397712 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:56:29.397775 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:29.397776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:29.397785 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:29.397840 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:56:29.398086 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.398695 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.399710 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.400117 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.400176 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:29.400183 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:29.400397 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 14:56:29.400449 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 14:56:29.400511 16117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.397573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:35443 (local address 127.30.74.66:50145)
0810 14:56:29.397637 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:29.397649 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:29.397698 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:29.397885 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:29.397888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:29.397895 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:56:29.397993 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:29.397996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.398817 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.398818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:29.399602 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:29.399605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.400212 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.400213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:29.400325 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:29.400328 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:56:29.400373 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 14:56:29.400406 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 14:56:29.400588 15966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.397765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:44215 (local address 127.30.74.65:37673)
0810 14:56:29.397831 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:29.397833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:29.398020 (+ 187us) server_negotiation.cc:407] Connection header received
0810 14:56:29.398052 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:29.398054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:29.398066 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:29.398120 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:56:29.398316 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.398806 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.399776 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.400130 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.400225 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:29.400233 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:29.400474 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 14:56:29.400517 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 14:56:29.400624 16284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.397693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:37673 (local address 127.30.74.66:44215)
0810 14:56:29.397918 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:29.397926 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:29.397977 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:29.398170 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:29.398171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:29.398173 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:56:29.398238 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:29.398240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.398893 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.398893 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:29.399671 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:29.399674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.400249 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.400289 (+ 40us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:29.400379 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:29.400384 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:56:29.400445 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 14:56:29.400481 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":10,"thread_start_us":132,"threads_started":1}
I20260810 14:56:29.400929 16214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "178b91bf8b9d4ec39c27aa5787f858dd" candidate_uuid: "f15189643c4542a6bea41b24510fe0c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36e5f7583b64495bcdf11bc656ce923" is_pre_election: true
I20260810 14:56:29.400924 15903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "178b91bf8b9d4ec39c27aa5787f858dd" candidate_uuid: "f15189643c4542a6bea41b24510fe0c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" is_pre_election: true
I20260810 14:56:29.401098 15903 raft_consensus.cc:2454] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f15189643c4542a6bea41b24510fe0c6 in term 0.
I20260810 14:56:29.401098 16214 raft_consensus.cc:2454] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f15189643c4542a6bea41b24510fe0c6 in term 0.
I20260810 14:56:29.401350 15986 leader_election.cc:302] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0b3ea338c174c8c95aa3aa912ba6c3d, f15189643c4542a6bea41b24510fe0c6; no voters:
I20260810 14:56:29.401502 16282 raft_consensus.cc:2781] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:56:29.401562 16282 raft_consensus.cc:482] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:56:29.401597 16282 raft_consensus.cc:3037] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:56:29.402417 16282 raft_consensus.cc:504] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } }
I20260810 14:56:29.402601 16282 leader_election.cc:288] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [CANDIDATE]: Term 1 election: Requested vote from peers b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67:35443), b0b3ea338c174c8c95aa3aa912ba6c3d (127.30.74.65:37673)
I20260810 14:56:29.402925 15903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "178b91bf8b9d4ec39c27aa5787f858dd" candidate_uuid: "f15189643c4542a6bea41b24510fe0c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d"
I20260810 14:56:29.403050 15903 raft_consensus.cc:3037] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:56:29.403867 16214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "178b91bf8b9d4ec39c27aa5787f858dd" candidate_uuid: "f15189643c4542a6bea41b24510fe0c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b36e5f7583b64495bcdf11bc656ce923"
I20260810 14:56:29.403951 16214 raft_consensus.cc:3037] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:56:29.404633 15903 raft_consensus.cc:2454] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f15189643c4542a6bea41b24510fe0c6 in term 1.
I20260810 14:56:29.404814 15986 leader_election.cc:302] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0b3ea338c174c8c95aa3aa912ba6c3d, f15189643c4542a6bea41b24510fe0c6; no voters:
I20260810 14:56:29.404912 16282 raft_consensus.cc:2781] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:56:29.405083 16282 raft_consensus.cc:686] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [term 1 LEADER]: Becoming Leader. State: Replica: f15189643c4542a6bea41b24510fe0c6, State: Running, Role: LEADER
I20260810 14:56:29.405118 16214 raft_consensus.cc:2454] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f15189643c4542a6bea41b24510fe0c6 in term 1.
I20260810 14:56:29.405221 16282 consensus_queue.cc:237] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } }
I20260810 14:56:29.406141 15750 catalog_manager.cc:5495] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to f15189643c4542a6bea41b24510fe0c6 (127.30.74.66). New cstate: current_term: 1 leader_uuid: "f15189643c4542a6bea41b24510fe0c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } health_report { overall_health: UNKNOWN } } }
W20260810 14:56:29.408998 15958 tablet.cc:2369] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:56:29.443261 16269 tablet.cc:2369] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:56:29.455017 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.452000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39086 (local address 127.30.74.66:38435)
0810 14:56:29.452103 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:29.452106 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:29.452117 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:56:29.452206 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:29.452207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:29.452232 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:29.452347 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:56:29.452494 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.453149 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.454014 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:29.454305 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:29.454395 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:29.454592 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:56:29.454715 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:56:29.454821 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:29.454848 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 14:56:29.454873 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:56:29.457120 16214 raft_consensus.cc:1261] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923 [term 1 FOLLOWER]: Refusing update from remote peer f15189643c4542a6bea41b24510fe0c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:56:29.457125 15903 raft_consensus.cc:1261] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Refusing update from remote peer f15189643c4542a6bea41b24510fe0c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:56:29.457489 16282 consensus_queue.cc:1035] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:56:29.457633 16285 consensus_queue.cc:1035] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:56:29.460322 16291 mvcc.cc:205] Tried to move back new op lower bound from 7316987041613373440 to 7316987041404403712. Current Snapshot: MvccSnapshot[applied={T|T < 7316987041613373440}]
I20260810 14:56:29.460443 16290 mvcc.cc:205] Tried to move back new op lower bound from 7316987041613373440 to 7316987041404403712. Current Snapshot: MvccSnapshot[applied={T|T < 7316987041613373440}]
I20260810 14:56:29.463150 15750 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:56:29.463279 15750 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:56:29.464178 15750 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Sending DeleteTablet for 3 replicas of tablet 178b91bf8b9d4ec39c27aa5787f858dd
I20260810 14:56:29.464582 15883 tablet_service.cc:1514] Processing DeleteTablet for tablet 178b91bf8b9d4ec39c27aa5787f858dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:56:29 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:34774
I20260810 14:56:29.464636 16039 tablet_service.cc:1514] Processing DeleteTablet for tablet 178b91bf8b9d4ec39c27aa5787f858dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:56:29 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:39070
I20260810 14:56:29.464656 16193 tablet_service.cc:1514] Processing DeleteTablet for tablet 178b91bf8b9d4ec39c27aa5787f858dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:56:29 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:48264
I20260810 14:56:29.464844 16300 tablet_replica.cc:330] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d: stopping tablet replica
I20260810 14:56:29.464906 16300 raft_consensus.cc:2229] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:56:29.464957 16300 raft_consensus.cc:2258] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:56:29.465010 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 15793
I20260810 14:56:29.472492 16302 tablet_replica.cc:330] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6: stopping tablet replica
I20260810 14:56:29.472687 16302 raft_consensus.cc:2229] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:56:29.472872 16302 raft_consensus.cc:2258] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:56:29.472892 16301 tablet_replica.cc:330] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923: stopping tablet replica
I20260810 14:56:29.473136 16301 raft_consensus.cc:2229] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:56:29.473317 16301 raft_consensus.cc:2258] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:56:29.473501 16302 ts_tablet_manager.cc:1905] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 14:56:29.473898 15734 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:37673 (error 108)
I20260810 14:56:29.474251 16301 ts_tablet_manager.cc:1905] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:56:29.474597 15965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.474454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:37673 (local address 127.0.0.1:34778)
0810 14:56:29.474509 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:29.474551 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:37673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 14:56:29.474720 15734 catalog_manager.cc:4584] TS b0b3ea338c174c8c95aa3aa912ba6c3d (127.30.74.65:37673): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 178b91bf8b9d4ec39c27aa5787f858dd: Network error: Client connection negotiation failed: client connection to 127.30.74.65:37673: connect: Connection refused (error 111)
I20260810 14:56:29.475399 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 15962
I20260810 14:56:29.475622 16302 ts_tablet_manager.cc:1918] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:56:29.475703 16302 log.cc:1196] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/178b91bf8b9d4ec39c27aa5787f858dd
I20260810 14:56:29.476423 16301 ts_tablet_manager.cc:1918] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:56:29.476567 16301 log.cc:1196] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/178b91bf8b9d4ec39c27aa5787f858dd
I20260810 14:56:29.476945 16301 ts_tablet_manager.cc:1939] T 178b91bf8b9d4ec39c27aa5787f858dd P b36e5f7583b64495bcdf11bc656ce923: Deleting consensus metadata
I20260810 14:56:29.477448 15735 catalog_manager.cc:4841] TS b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67:35443): tablet 178b91bf8b9d4ec39c27aa5787f858dd (table test-table [id=f9830fc27d144f5aaf773ab8a830cd3a]) successfully deleted
I20260810 14:56:29.482200 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 16118
I20260810 14:56:29.482230 15965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:29.482105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:38435 (local address 127.0.0.1:39100)
0810 14:56:29.482160 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:29.482196 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.66:38435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 14:56:29.489225 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 15714
I20260810 14:56:49.497327 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:39299
--webserver_interface=127.30.74.126
--webserver_port=40493
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:39299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:56:49.581218 16304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:56:49.581450 16304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:56:49.581475 16304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:56:49.582669 16304 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 14:56:49.582715 16304 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 14:56:49.582727 16304 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 14:56:49.582739 16304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:56:49.582796 16304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:56:49.582816 16304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:56:49.582839 16304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:56:49.582883 16304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:56:49.584527 16304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:39299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:39299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=40493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:56:49.584800 16304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:56:49.585049 16304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:56:49.590502 16309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:49.590509 16310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:49.590551 16304 server_base.cc:1034] running on GCE node
W20260810 14:56:49.590509 16313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:49.591169 16304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:56:49.591549 16304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:56:49.592736 16304 hybrid_clock.cc:648] HybridClock initialized: now 1786373809592726 us; error 41 us; skew 500 ppm
Aug 10 14:56:49 dist-test-slave-0b58 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373809, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:56:49.595477 16304 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:56:49.595975 16304 webserver.cc:460] Webserver started at http://127.30.74.126:40493/ using document root <none> and password file <none>
I20260810 14:56:49.596215 16304 fs_manager.cc:359] Metadata directory not provided
I20260810 14:56:49.596318 16304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:56:49.597790 16304 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:56:49.598568 16321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:49.598809 16304 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:56:49.598932 16304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "7cf71e9ce81340fd9b668d7233c75a59"
format_stamp: "Formatted at 2026-08-10 14:56:07 on dist-test-slave-0b58"
server_key: "1ef860246939163d763c88364019d165"
server_key_iv: "5c9339b1725aaa597c3fcbaaf34bdf9a"
server_key_version: "encryptionkey@0"
I20260810 14:56:49.599324 16304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:56:49.609824 16304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:56:49.612004 16304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:56:49.612206 16304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:56:49.616403 16304 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:39299
I20260810 14:56:49.616448 16385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:39299 every 8 connection(s)
I20260810 14:56:49.616859 16304 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:56:49.618328 16386 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:56:49.619040 16386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Bootstrap starting.
I20260810 14:56:49.623662 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 16304
I20260810 14:56:49.624087 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:37673
--local_ip_for_outbound_sockets=127.30.74.65
--tserver_master_addrs=127.30.74.126:39299
--webserver_port=34273
--webserver_interface=127.30.74.65
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:56:49.628080 16386 log.cc:824] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Log is configured to *not* fsync() on all Append() calls
I20260810 14:56:49.630738 16386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:56:49.631102 16386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Bootstrap complete.
I20260810 14:56:49.633914 16386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } }
I20260810 14:56:49.634176 16386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cf71e9ce81340fd9b668d7233c75a59, State: Initialized, Role: FOLLOWER
I20260810 14:56:49.634390 16386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } }
I20260810 14:56:49.634459 16386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:56:49.634528 16386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:56:49.634584 16386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 2 FOLLOWER]: Advancing to term 3
I20260810 14:56:49.635744 16386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } }
I20260810 14:56:49.635843 16386 leader_election.cc:302] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cf71e9ce81340fd9b668d7233c75a59; no voters:
I20260810 14:56:49.636092 16386 leader_election.cc:288] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 14:56:49.636139 16390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 14:56:49.636458 16386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:56:49.636500 16390 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [term 3 LEADER]: Becoming Leader. State: Replica: 7cf71e9ce81340fd9b668d7233c75a59, State: Running, Role: LEADER
I20260810 14:56:49.636770 16390 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } }
I20260810 14:56:49.637137 16392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cf71e9ce81340fd9b668d7233c75a59. Latest consensus state: current_term: 3 leader_uuid: "7cf71e9ce81340fd9b668d7233c75a59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } } }
I20260810 14:56:49.637148 16391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7cf71e9ce81340fd9b668d7233c75a59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf71e9ce81340fd9b668d7233c75a59" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39299 } } }
I20260810 14:56:49.637249 16392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [sys.catalog]: This master's current role is: LEADER
I20260810 14:56:49.637269 16391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59 [sys.catalog]: This master's current role is: LEADER
I20260810 14:56:49.637470 16400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:56:49.638401 16400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:56:49.638527 16400 catalog_manager.cc:1259] Loaded cluster ID: 5c97056981314348afe052dec8c4b9d9
I20260810 14:56:49.638571 16400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:56:49.639937 16400 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:56:49.640198 16400 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Loaded TSK: 0
I20260810 14:56:49.640295 16400 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7cf71e9ce81340fd9b668d7233c75a59: Loaded TSK: 1
F20260810 14:56:49.640565 16400 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc334f1e9ad google::LogMessage::Fail() at ??:0
@ 0x7fc334f2188a google::LogMessage::SendToLog() at ??:0
@ 0x7fc334f1e4de google::LogMessage::Flush() at ??:0
@ 0x7fc334f1fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc33768b104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fc337688d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fc33ce7c1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fc33ce7c48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fc33ce7c594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fc33ce42c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fc33ce4cad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc336dec0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc336de1954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc337c456db start_thread at ??:0
@ 0x7fc333a1c61f clone at ??:0
W20260810 14:56:49.735247 16388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:56:49.735535 16388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:56:49.735576 16388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:56:49.737402 16388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:56:49.737469 16388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:56:49.737512 16388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:56:49.739907 16388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:37673
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=34273
--tserver_master_addrs=127.30.74.126:39299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:56:49.740221 16388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:56:49.740533 16388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:56:49.746068 16435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:49.746163 16433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:49.746321 16388 server_base.cc:1034] running on GCE node
W20260810 14:56:49.746081 16432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:49.746594 16388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:56:49.746865 16388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:56:49.748066 16388 hybrid_clock.cc:648] HybridClock initialized: now 1786373809748039 us; error 44 us; skew 500 ppm
Aug 10 14:56:49 dist-test-slave-0b58 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373809, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:56:49.750998 16388 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:56:49.751516 16388 webserver.cc:460] Webserver started at http://127.30.74.65:34273/ using document root <none> and password file <none>
I20260810 14:56:49.751809 16388 fs_manager.cc:359] Metadata directory not provided
I20260810 14:56:49.751891 16388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:56:49.753520 16388 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:56:49.754365 16443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:49.754570 16388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:56:49.754689 16388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d"
format_stamp: "Formatted at 2026-08-10 14:56:07 on dist-test-slave-0b58"
server_key: "277cef01196f4730b51d9d3ae3a21d6c"
server_key_iv: "6cea70d52a56a68cb0082ca632a68fe7"
server_key_version: "encryptionkey@0"
I20260810 14:56:49.755091 16388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:56:49.769158 16388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:56:49.771593 16388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:56:49.771775 16388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:56:49.772025 16388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:56:49.772584 16450 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 14:56:49.774660 16388 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 14:56:49.774747 16388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 14:56:49.774797 16388 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 14:56:49.775455 16388 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 14:56:49.775512 16388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 14:56:49.775593 16450 tablet_bootstrap.cc:492] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d: Bootstrap starting.
I20260810 14:56:49.783088 16388 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:37673
I20260810 14:56:49.783145 16567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:37673 every 8 connection(s)
I20260810 14:56:49.783548 16388 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:56:49.786996 16450 log.cc:824] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:56:49.789929 16450 tablet_bootstrap.cc:492] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:56:49.790279 16450 tablet_bootstrap.cc:492] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d: Bootstrap complete.
I20260810 14:56:49.791136 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 16388
I20260810 14:56:49.791510 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:38435
--local_ip_for_outbound_sockets=127.30.74.66
--tserver_master_addrs=127.30.74.126:39299
--webserver_port=45401
--webserver_interface=127.30.74.66
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:56:49.791553 16450 ts_tablet_manager.cc:1397] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.010s
I20260810 14:56:49.793064 16450 raft_consensus.cc:348] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } }
I20260810 14:56:49.793414 16450 raft_consensus.cc:729] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0b3ea338c174c8c95aa3aa912ba6c3d, State: Initialized, Role: FOLLOWER
I20260810 14:56:49.793570 16450 consensus_queue.cc:260] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } }
I20260810 14:56:49.793862 16450 ts_tablet_manager.cc:1428] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d: Time spent starting tablet: real 0.002s user 0.003s sys 0.002s
W20260810 14:56:49.898505 16570 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:56:49.784402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:44943)
0810 14:56:49.784582 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:49.784635 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:49.785092 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:49.898243 (+113151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
W20260810 14:56:49.898813 16568 heartbeater.cc:644] Failed to heartbeat to 127.30.74.126:39299 (0 consecutive failures): Network error: Failed to ping master at 127.30.74.126:39299: Client connection negotiation failed: client connection to 127.30.74.126:39299: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 14:56:49.899171 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:49.899050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:39479)
0810 14:56:49.899105 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:49.899145 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 14:56:49.899539 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:49.899454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:53051)
0810 14:56:49.899500 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:49.899524 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 14:56:49.899320 16573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:56:49.899623 16573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:56:49.899677 16573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:56:49.899669 16568 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:56:49.901042 16573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:56:49.901089 16573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:56:49.901125 16573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:56:49.902709 16573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:38435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=45401
--tserver_master_addrs=127.30.74.126:39299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:56:49.902967 16573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:56:49.903223 16573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:56:49.908604 16584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:49.908615 16582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:49.908907 16573 server_base.cc:1034] running on GCE node
W20260810 14:56:49.908958 16581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:49.909186 16573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:56:49.909464 16573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:56:49.910648 16573 hybrid_clock.cc:648] HybridClock initialized: now 1786373809910629 us; error 43 us; skew 500 ppm
Aug 10 14:56:49 dist-test-slave-0b58 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373809, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:56:49.913800 16573 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:56:49.914387 16573 webserver.cc:460] Webserver started at http://127.30.74.66:45401/ using document root <none> and password file <none>
I20260810 14:56:49.914633 16573 fs_manager.cc:359] Metadata directory not provided
I20260810 14:56:49.914706 16573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:56:49.916177 16573 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:56:49.916996 16591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:49.917201 16573 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:56:49.917336 16573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "f15189643c4542a6bea41b24510fe0c6"
format_stamp: "Formatted at 2026-08-10 14:56:08 on dist-test-slave-0b58"
server_key: "fb5cedba153838e03cdf33f71613e17a"
server_key_iv: "649d3e08427b376ab6b6e2d661091228"
server_key_version: "encryptionkey@0"
I20260810 14:56:49.917711 16573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:56:49.944777 16573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:56:49.946846 16573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:56:49.947021 16573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:56:49.947309 16573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:56:49.947800 16598 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260810 14:56:49.949280 16598 ts_tablet_manager.cc:1852] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260810 14:56:49.949574 16598 ts_tablet_manager.cc:1905] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:56:49.951885 16598 ts_tablet_manager.cc:1918] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:56:49.951964 16598 log.cc:1196] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/178b91bf8b9d4ec39c27aa5787f858dd
I20260810 14:56:49.952105 16598 ts_tablet_manager.cc:1939] T 178b91bf8b9d4ec39c27aa5787f858dd P f15189643c4542a6bea41b24510fe0c6: Deleting consensus metadata
I20260810 14:56:49.952396 16573 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260810 14:56:49.952481 16573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260810 14:56:49.952531 16573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:56:49.952565 16573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:49.959192 16573 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:38435
I20260810 14:56:49.959280 16720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:38435 every 8 connection(s)
I20260810 14:56:49.959625 16573 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:56:49.960851 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:49.960452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:32867)
0810 14:56:49.960629 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:49.960704 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":74,"threads_started":1}
W20260810 14:56:49.961123 16721 heartbeater.cc:644] Failed to heartbeat to 127.30.74.126:39299 (0 consecutive failures): Network error: Failed to ping master at 127.30.74.126:39299: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
I20260810 14:56:49.961510 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:49.961366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:40581)
0810 14:56:49.961453 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:49.961486 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 14:56:49.961877 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:49.961793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:51083)
0810 14:56:49.961835 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:49.961860 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 14:56:49.961999 16721 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 14:56:49.969748 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 16573
I20260810 14:56:49.970090 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:35443
--local_ip_for_outbound_sockets=127.30.74.67
--tserver_master_addrs=127.30.74.126:39299
--webserver_port=45565
--webserver_interface=127.30.74.67
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:56:50.057593 16724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:56:50.057822 16724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:56:50.057848 16724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:56:50.059085 16724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:56:50.059131 16724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:56:50.059159 16724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:56:50.060745 16724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:35443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=45565
--tserver_master_addrs=127.30.74.126:39299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:56:50.061038 16724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:56:50.061296 16724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:56:50.066721 16732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:50.066830 16724 server_base.cc:1034] running on GCE node
W20260810 14:56:50.066807 16729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:56:50.066721 16730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:56:50.067176 16724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:56:50.067478 16724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:56:50.068657 16724 hybrid_clock.cc:648] HybridClock initialized: now 1786373810068627 us; error 41 us; skew 500 ppm
Aug 10 14:56:50 dist-test-slave-0b58 krb5kdc[15060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373810, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:56:50.071388 16724 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:56:50.071928 16724 webserver.cc:460] Webserver started at http://127.30.74.67:45565/ using document root <none> and password file <none>
I20260810 14:56:50.072197 16724 fs_manager.cc:359] Metadata directory not provided
I20260810 14:56:50.072248 16724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:56:50.073741 16724 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:56:50.074517 16741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:50.074772 16724 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:56:50.074895 16724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "b36e5f7583b64495bcdf11bc656ce923"
format_stamp: "Formatted at 2026-08-10 14:56:08 on dist-test-slave-0b58"
server_key: "3ba740b356b172cf85845c4be1128584"
server_key_iv: "41868100cd521213b62210db9df26a52"
server_key_version: "encryptionkey@0"
I20260810 14:56:50.075282 16724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:56:50.113945 16724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:56:50.116550 16724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:56:50.116744 16724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:56:50.117003 16724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:56:50.117331 16724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:56:50.117388 16724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:50.117434 16724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:56:50.117465 16724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:56:50.123764 16724 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:35443
I20260810 14:56:50.123816 16865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:35443 every 8 connection(s)
I20260810 14:56:50.124191 16724 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:56:50.125342 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:50.124959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:41579)
0810 14:56:50.125113 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:50.125188 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
W20260810 14:56:50.125757 16866 heartbeater.cc:644] Failed to heartbeat to 127.30.74.126:39299 (0 consecutive failures): Network error: Failed to ping master at 127.30.74.126:39299: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
I20260810 14:56:50.126076 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:50.125979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:37931)
0810 14:56:50.126027 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:50.126057 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 14:56:50.126645 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:50.126504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:58317)
0810 14:56:50.126578 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:50.126609 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 14:56:50.126837 16866 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 14:56:50.127005 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 16724
I20260810 14:56:50.900462 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:50.900090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:49569)
0810 14:56:50.900338 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:50.900393 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":74,"threads_started":1}
I20260810 14:56:50.962563 16871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:50.962363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:34857)
0810 14:56:50.962485 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:50.962523 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260810 14:56:51.051364 16872 raft_consensus.cc:482] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:56:51.051494 16872 raft_consensus.cc:504] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } }
I20260810 14:56:51.051998 16872 leader_election.cc:288] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67:35443), f15189643c4542a6bea41b24510fe0c6 (127.30.74.66:38435)
Aug 10 14:56:51 dist-test-slave-0b58 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373809, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.67@KRBTEST.COM
Aug 10 14:56:51 dist-test-slave-0b58 krb5kdc[15060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373809, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.66@KRBTEST.COM
I20260810 14:56:51.065538 16875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:51.052611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:59535 (local address 127.30.74.66:38435)
0810 14:56:51.052789 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:51.052798 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:51.052821 (+ 23us) server_negotiation.cc:407] Connection header received
0810 14:56:51.053471 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:51.053474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:51.053518 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:51.053631 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:56:51.054623 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:51.056019 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:51.057633 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:51.058272 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:51.058394 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:51.061729 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:56:51.061733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:56:51.061733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:56:51.061744 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:56:51.063855 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:51.064303 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:51.064306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:51.064306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:51.064418 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:51.064664 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:51.064665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:51.064666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:51.064925 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:56:51.065009 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:51.065271 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 14:56:51.065407 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260810 14:56:51.065522 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:51.052243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:35443 (local address 127.30.74.65:47841)
0810 14:56:51.052340 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:51.052352 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:51.052414 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:51.053693 (+ 1279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:51.053696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:51.054183 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:56:51.054485 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:51.054493 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:51.056149 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:51.056150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:51.057478 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:51.057481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:51.058382 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:51.058384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:51.058541 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:51.059275 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:56:51.059286 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:56:51.061321 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:56:51.063579 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:56:51.063587 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:56:51.063590 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:56:51.063951 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:56:51.064293 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:56:51.064294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:56:51.064294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:56:51.064430 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:56:51.064923 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:56:51.064931 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:56:51.065077 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 14:56:51.065207 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 14:56:51.065406 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":121}
I20260810 14:56:51.065569 16874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:51.052442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:47841 (local address 127.30.74.67:35443)
0810 14:56:51.052620 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 14:56:51.052631 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:56:51.052822 (+ 191us) server_negotiation.cc:407] Connection header received
0810 14:56:51.053374 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:56:51.053377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:56:51.053453 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:56:51.053618 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:56:51.054622 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:51.056042 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:51.057615 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:56:51.058260 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:56:51.058394 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:51.061462 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:56:51.061468 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:56:51.061468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:56:51.061483 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:56:51.063495 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:51.064063 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:51.064067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:51.064068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:51.064188 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:56:51.064530 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:56:51.064531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:56:51.064531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:56:51.064847 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:56:51.064912 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 14:56:51.065289 (+ 377us) server_negotiation.cc:299] Negotiation successful
0810 14:56:51.065446 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
I20260810 14:56:51.065636 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:51.052417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:38435 (local address 127.30.74.65:59535)
0810 14:56:51.052628 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:51.052641 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:56:51.052715 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:56:51.053693 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:56:51.053696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:56:51.054217 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:56:51.054485 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:51.054493 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:51.056164 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:51.056164 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:51.057478 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:56:51.057481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:56:51.058383 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:56:51.058384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:56:51.058541 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:56:51.059275 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:56:51.059286 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:56:51.061598 (+ 2312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:56:51.063945 (+ 2347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:56:51.063946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:56:51.063946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:56:51.064193 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:56:51.064521 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:56:51.064523 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:56:51.064523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:56:51.064595 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:56:51.065011 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:56:51.065012 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:56:51.065076 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 14:56:51.065187 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 14:56:51.065487 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":177,"thread_start_us":57,"threads_started":1}
W20260810 14:56:51.066459 16446 leader_election.cc:341] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67:35443): Not found: Tablet not found: 178b91bf8b9d4ec39c27aa5787f858dd
W20260810 14:56:51.066994 16446 leader_election.cc:341] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f15189643c4542a6bea41b24510fe0c6 (127.30.74.66:38435): Not found: Tablet not found: 178b91bf8b9d4ec39c27aa5787f858dd
I20260810 14:56:51.067065 16446 leader_election.cc:302] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0b3ea338c174c8c95aa3aa912ba6c3d; no voters: b36e5f7583b64495bcdf11bc656ce923, f15189643c4542a6bea41b24510fe0c6
I20260810 14:56:51.067207 16872 raft_consensus.cc:2726] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:56:51.127478 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:51.127190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:35019)
0810 14:56:51.127355 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:51.127432 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260810 14:56:51.901340 16877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:51.901058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:58519)
0810 14:56:51.901243 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:51.901293 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":75,"threads_started":1}
I20260810 14:56:51.963385 16878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:51.963118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:52051)
0810 14:56:51.963276 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:51.963338 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260810 14:56:52.128373 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:52.128041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:56967)
0810 14:56:52.128193 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:52.128239 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
I20260810 14:56:52.743871 16880 raft_consensus.cc:482] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:56:52.743973 16880 raft_consensus.cc:504] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } }
I20260810 14:56:52.744138 16880 leader_election.cc:288] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67:35443), f15189643c4542a6bea41b24510fe0c6 (127.30.74.66:38435)
W20260810 14:56:52.744673 16446 leader_election.cc:341] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67:35443): Not found: Tablet not found: 178b91bf8b9d4ec39c27aa5787f858dd
W20260810 14:56:52.744768 16446 leader_election.cc:341] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f15189643c4542a6bea41b24510fe0c6 (127.30.74.66:38435): Not found: Tablet not found: 178b91bf8b9d4ec39c27aa5787f858dd
I20260810 14:56:52.744802 16446 leader_election.cc:302] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0b3ea338c174c8c95aa3aa912ba6c3d; no voters: b36e5f7583b64495bcdf11bc656ce923, f15189643c4542a6bea41b24510fe0c6
I20260810 14:56:52.744886 16880 raft_consensus.cc:2726] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:56:52.902091 16881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:52.901837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:40495)
0810 14:56:52.901998 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:52.902040 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":71,"threads_started":1}
I20260810 14:56:52.964304 16882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:52.964017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:35017)
0810 14:56:52.964166 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:52.964215 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":61,"threads_started":1}
I20260810 14:56:53.129280 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:53.128942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:50341)
0810 14:56:53.129190 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:53.129233 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":67,"threads_started":1}
I20260810 14:56:53.902988 16885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:53.902688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:59915)
0810 14:56:53.902850 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:53.902902 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":109,"threads_started":1}
I20260810 14:56:53.965238 16886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:53.964954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:46305)
0810 14:56:53.965129 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:53.965183 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":72,"threads_started":1}
I20260810 14:56:54.130110 16887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:54.129892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:43901)
0810 14:56:54.130019 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:54.130061 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260810 14:56:54.800390 16888 raft_consensus.cc:482] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:56:54.800501 16888 raft_consensus.cc:504] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } }
I20260810 14:56:54.800812 16888 leader_election.cc:288] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67:35443), f15189643c4542a6bea41b24510fe0c6 (127.30.74.66:38435)
W20260810 14:56:54.801446 16446 leader_election.cc:341] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67:35443): Not found: Tablet not found: 178b91bf8b9d4ec39c27aa5787f858dd
W20260810 14:56:54.801561 16446 leader_election.cc:341] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f15189643c4542a6bea41b24510fe0c6 (127.30.74.66:38435): Not found: Tablet not found: 178b91bf8b9d4ec39c27aa5787f858dd
I20260810 14:56:54.801607 16446 leader_election.cc:302] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0b3ea338c174c8c95aa3aa912ba6c3d; no voters: b36e5f7583b64495bcdf11bc656ce923, f15189643c4542a6bea41b24510fe0c6
I20260810 14:56:54.801707 16888 raft_consensus.cc:2726] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:56:54.903949 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:54.903625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:41529)
0810 14:56:54.903844 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:54.903895 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":96,"threads_started":1}
I20260810 14:56:54.966183 16890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:54.965922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:59187)
0810 14:56:54.966075 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:54.966133 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
I20260810 14:56:55.131112 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:55.130823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:58005)
0810 14:56:55.131012 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:55.131061 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":86,"threads_started":1}
I20260810 14:56:55.905169 16892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:55.904881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:41907)
0810 14:56:55.905061 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:55.905119 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":74,"threads_started":1}
I20260810 14:56:55.967080 16893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:55.966816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:38205)
0810 14:56:55.966980 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:55.967032 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
I20260810 14:56:56.131991 16894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:56.131722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:52289)
0810 14:56:56.131856 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:56.131920 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260810 14:56:56.906145 16895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:56.905883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:52237)
0810 14:56:56.906039 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:56.906088 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":71,"threads_started":1}
I20260810 14:56:56.967929 16896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:56.967655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:36845)
0810 14:56:56.967839 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:56.967880 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":70,"threads_started":1}
I20260810 14:56:57.133015 16897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:57.132699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:57999)
0810 14:56:57.132905 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:57.132961 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":99,"threads_started":1}
I20260810 14:56:57.907074 16898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:57.906786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:35741)
0810 14:56:57.906973 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:57.907027 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":71,"threads_started":1}
I20260810 14:56:57.968953 16899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:57.968671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:57163)
0810 14:56:57.968850 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:57.968902 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":78,"threads_started":1}
I20260810 14:56:58.133883 16900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:58.133632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:52521)
0810 14:56:58.133778 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:58.133834 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":70,"threads_started":1}
I20260810 14:56:58.907989 16902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:58.907721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:40221)
0810 14:56:58.907887 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:58.907939 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":68,"threads_started":1}
I20260810 14:56:58.969857 16903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:58.969587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:54255)
0810 14:56:58.969756 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:58.969807 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":80,"threads_started":1}
I20260810 14:56:59.134841 16904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:59.134549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:45971)
0810 14:56:59.134735 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:59.134789 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":89,"threads_started":1}
I20260810 14:56:59.159549 16905 raft_consensus.cc:482] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:56:59.159657 16905 raft_consensus.cc:504] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } }
I20260810 14:56:59.159989 16905 leader_election.cc:288] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67:35443), f15189643c4542a6bea41b24510fe0c6 (127.30.74.66:38435)
W20260810 14:56:59.160450 16446 leader_election.cc:341] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67:35443): Not found: Tablet not found: 178b91bf8b9d4ec39c27aa5787f858dd
W20260810 14:56:59.160528 16446 leader_election.cc:341] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f15189643c4542a6bea41b24510fe0c6 (127.30.74.66:38435): Not found: Tablet not found: 178b91bf8b9d4ec39c27aa5787f858dd
I20260810 14:56:59.160547 16446 leader_election.cc:302] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0b3ea338c174c8c95aa3aa912ba6c3d; no voters: b36e5f7583b64495bcdf11bc656ce923, f15189643c4542a6bea41b24510fe0c6
I20260810 14:56:59.160615 16905 raft_consensus.cc:2726] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:56:59.908905 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:59.908625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:33357)
0810 14:56:59.908805 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:59.908857 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":72,"threads_started":1}
I20260810 14:56:59.970698 16908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:56:59.970438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:44989)
0810 14:56:59.970595 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 14:56:59.970645 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260810 14:57:00.135785 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:00.135495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:36443)
0810 14:57:00.135685 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:00.135735 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":76,"threads_started":1}
I20260810 14:57:00.909983 16910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:00.909618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:50637)
0810 14:57:00.909864 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:00.909928 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":85,"threads_started":1}
I20260810 14:57:00.971642 16911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:00.971346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:52247)
0810 14:57:00.971528 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:00.971579 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
I20260810 14:57:01.136785 16913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:01.136480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:49535)
0810 14:57:01.136650 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:01.136732 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":84,"threads_started":1}
I20260810 14:57:01.745038 16914 raft_consensus.cc:482] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:57:01.745151 16914 raft_consensus.cc:504] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b36e5f7583b64495bcdf11bc656ce923" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35443 } } peers { permanent_uuid: "f15189643c4542a6bea41b24510fe0c6" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38435 } } peers { permanent_uuid: "b0b3ea338c174c8c95aa3aa912ba6c3d" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37673 } }
I20260810 14:57:01.745355 16914 leader_election.cc:288] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67:35443), f15189643c4542a6bea41b24510fe0c6 (127.30.74.66:38435)
W20260810 14:57:01.746013 16446 leader_election.cc:341] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b36e5f7583b64495bcdf11bc656ce923 (127.30.74.67:35443): Not found: Tablet not found: 178b91bf8b9d4ec39c27aa5787f858dd
W20260810 14:57:01.746129 16446 leader_election.cc:341] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f15189643c4542a6bea41b24510fe0c6 (127.30.74.66:38435): Not found: Tablet not found: 178b91bf8b9d4ec39c27aa5787f858dd
I20260810 14:57:01.746183 16446 leader_election.cc:302] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0b3ea338c174c8c95aa3aa912ba6c3d; no voters: b36e5f7583b64495bcdf11bc656ce923, f15189643c4542a6bea41b24510fe0c6
I20260810 14:57:01.746276 16914 raft_consensus.cc:2726] T 178b91bf8b9d4ec39c27aa5787f858dd P b0b3ea338c174c8c95aa3aa912ba6c3d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:57:01.911054 16915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:01.910728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:41121)
0810 14:57:01.910949 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:01.911004 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":97,"threads_started":1}
I20260810 14:57:01.972739 16916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:01.972448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:37871)
0810 14:57:01.972630 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:01.972683 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":87,"threads_started":1}
I20260810 14:57:02.137791 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:02.137446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:49989)
0810 14:57:02.137663 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:02.137717 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":87,"threads_started":1}
I20260810 14:57:02.912098 16918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:02.911808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:39615)
0810 14:57:02.911997 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:02.912049 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":87,"threads_started":1}
I20260810 14:57:02.973596 16919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:02.973318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:54221)
0810 14:57:02.973504 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:02.973539 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":94,"threads_started":1}
I20260810 14:57:03.138836 16920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:03.138530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:32893)
0810 14:57:03.138735 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:03.138786 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":97,"threads_started":1}
I20260810 14:57:03.913062 16921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:03.912777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:51845)
0810 14:57:03.912955 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:03.913012 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":90,"threads_started":1}
I20260810 14:57:03.974586 16922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:03.974300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:56311)
0810 14:57:03.974488 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:03.974537 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":90,"threads_started":1}
I20260810 14:57:04.139910 16923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:04.139624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:54447)
0810 14:57:04.139800 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:04.139851 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":79,"threads_started":1}
I20260810 14:57:04.914325 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:04.913992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.65:52051)
0810 14:57:04.914215 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:04.914276 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":160,"threads_started":1}
I20260810 14:57:04.975742 16925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:04.975451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.66:35891)
0810 14:57:04.975630 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:04.975692 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":90,"threads_started":1}
I20260810 14:57:05.132687 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 16388
I20260810 14:57:05.141276 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:05.140800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39299 (local address 127.30.74.67:34531)
0810 14:57:05.141126 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:05.141193 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.126:39299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":239,"threads_started":1}
I20260810 14:57:05.144006 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 16573
I20260810 14:57:05.152158 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 16724
2026-08-10T14:57:05Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58749 ms)
[----------] 30 tests from SecurityITest (238638 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:05 dist-test-slave-0b58 krb5kdc[16929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:05 dist-test-slave-0b58 krb5kdc[16929](info): set up 2 sockets
Aug 10 14:57:05 dist-test-slave-0b58 krb5kdc[16929](info): commencing operation
krb5kdc: starting...
W20260810 14:57:06.336849 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:06 dist-test-slave-0b58 krb5kdc[16929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:57:06.492470 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:42263
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:42263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:06.580200 16944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:06.580502 16944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:06.580544 16944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:06.581810 16944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:06.581892 16944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:06.581918 16944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:06.581938 16944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:06.581969 16944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:06.583595 16944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:42263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:42263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:06.583925 16944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:06.584204 16944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 14:57:06.590919 16944 server_base.cc:1034] running on GCE node
W20260810 14:57:06.590893 16952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:06.590893 16949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:06.591229 16950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:06.591681 16944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:06.592025 16944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:06.593202 16944 hybrid_clock.cc:648] HybridClock initialized: now 1786373826593179 us; error 51 us; skew 500 ppm
Aug 10 14:57:06 dist-test-slave-0b58 krb5kdc[16929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373826, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:06.596325 16944 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:57:06.596853 16944 webserver.cc:460] Webserver started at http://127.30.74.126:33137/ using document root <none> and password file <none>
I20260810 14:57:06.597133 16944 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:06.597205 16944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:06.597396 16944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:06.598541 16944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "b36b6d3b3ccf442395a589f1113af5cb"
format_stamp: "Formatted at 2026-08-10 14:57:06 on dist-test-slave-0b58"
server_key: "24b2368354c3a8fa11313121fb180f7c"
server_key_iv: "6a5408072e741780b8b156ceca68ec31"
server_key_version: "encryptionkey@0"
I20260810 14:57:06.598937 16944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "b36b6d3b3ccf442395a589f1113af5cb"
format_stamp: "Formatted at 2026-08-10 14:57:06 on dist-test-slave-0b58"
server_key: "24b2368354c3a8fa11313121fb180f7c"
server_key_iv: "6a5408072e741780b8b156ceca68ec31"
server_key_version: "encryptionkey@0"
I20260810 14:57:06.600679 16944 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 14:57:06.601959 16962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:06.602434 16944 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:06.602665 16944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "b36b6d3b3ccf442395a589f1113af5cb"
format_stamp: "Formatted at 2026-08-10 14:57:06 on dist-test-slave-0b58"
server_key: "24b2368354c3a8fa11313121fb180f7c"
server_key_iv: "6a5408072e741780b8b156ceca68ec31"
server_key_version: "encryptionkey@0"
I20260810 14:57:06.602758 16944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:06.626884 16944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:06.629892 16944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:06.630095 16944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:06.634542 16944 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:42263
I20260810 14:57:06.634581 17020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:42263 every 8 connection(s)
I20260810 14:57:06.634984 16944 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:06.635941 17021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:06.638449 17021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb: Bootstrap starting.
I20260810 14:57:06.639039 17021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:06.639294 17021 log.cc:824] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:06.639911 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 16944
I20260810 14:57:06.640035 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:06.640182 17021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb: No bootstrap required, opened a new log
I20260810 14:57:06.640286 31017 external_mini_cluster.cc:1428] Setting key 0e981ca97ee982d03b1b1b0bd1322556
I20260810 14:57:06.643078 17021 raft_consensus.cc:348] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b36b6d3b3ccf442395a589f1113af5cb" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42263 } }
I20260810 14:57:06.643250 17021 raft_consensus.cc:374] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:06.643275 17021 raft_consensus.cc:729] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b36b6d3b3ccf442395a589f1113af5cb, State: Initialized, Role: FOLLOWER
I20260810 14:57:06.643421 17021 consensus_queue.cc:260] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b36b6d3b3ccf442395a589f1113af5cb" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42263 } }
I20260810 14:57:06.643505 17021 raft_consensus.cc:388] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:06.643563 17021 raft_consensus.cc:482] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:06.643615 17021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:06.644380 17021 raft_consensus.cc:504] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b36b6d3b3ccf442395a589f1113af5cb" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42263 } }
I20260810 14:57:06.644485 17021 leader_election.cc:302] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b36b6d3b3ccf442395a589f1113af5cb; no voters:
I20260810 14:57:06.644727 17021 leader_election.cc:288] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:06.644755 17026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:06.644946 17026 raft_consensus.cc:686] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [term 1 LEADER]: Becoming Leader. State: Replica: b36b6d3b3ccf442395a589f1113af5cb, State: Running, Role: LEADER
I20260810 14:57:06.645041 17026 consensus_queue.cc:237] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b36b6d3b3ccf442395a589f1113af5cb" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42263 } }
I20260810 14:57:06.645114 17021 sys_catalog.cc:564] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:06.645386 17026 sys_catalog.cc:455] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b36b6d3b3ccf442395a589f1113af5cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b36b6d3b3ccf442395a589f1113af5cb" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42263 } } }
I20260810 14:57:06.645555 17026 sys_catalog.cc:458] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:06.645803 17026 sys_catalog.cc:455] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b36b6d3b3ccf442395a589f1113af5cb. Latest consensus state: current_term: 1 leader_uuid: "b36b6d3b3ccf442395a589f1113af5cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b36b6d3b3ccf442395a589f1113af5cb" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 42263 } } }
I20260810 14:57:06.645947 17026 sys_catalog.cc:458] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:06.646345 17035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:06.646978 17035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:06.648998 17035 catalog_manager.cc:1347] Generated new cluster ID: f5ab26087b554ccb9eba7680bd32a413
I20260810 14:57:06.649063 17035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 14:57:06 dist-test-slave-0b58 krb5kdc[16929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:06.653419 17025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:06.640881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50486 (local address 127.30.74.126:42263)
0810 14:57:06.641181 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:06.641190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:06.641230 (+ 40us) server_negotiation.cc:407] Connection header received
0810 14:57:06.641795 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:06.641804 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:06.641877 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:06.642011 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:06.642960 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:06.644695 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:06.645879 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:06.646566 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:06.646653 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:06.649782 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:06.649787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:06.649787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:06.649800 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:06.651646 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:06.652088 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:06.652092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:06.652093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:06.652198 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:06.652469 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:06.652470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:06.652470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:06.652756 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:06.652946 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:06.653019 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 14:57:06.653177 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":120,"threads_started":1}
I20260810 14:57:06.657027 17035 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:06.657622 17035 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:06.662084 17035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb: Generated new TSK 0
I20260810 14:57:06.662344 17035 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:57:06.700009 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42263
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:06.795473 17050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:06.795753 17050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:06.795797 17050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:06.797230 17050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:06.797317 17050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:06.797366 17050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:57:06.799125 17050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:06.799454 17050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:06.799710 17050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:06.805482 17058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:06.805503 17055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:06.805680 17050 server_base.cc:1034] running on GCE node
W20260810 14:57:06.805485 17056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:06.806074 17050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:06.806420 17050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:06.807682 17050 hybrid_clock.cc:648] HybridClock initialized: now 1786373826807655 us; error 43 us; skew 500 ppm
Aug 10 14:57:06 dist-test-slave-0b58 krb5kdc[16929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373826, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:06.810699 17050 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:57:06.811244 17050 webserver.cc:460] Webserver started at http://127.30.74.65:37159/ using document root <none> and password file <none>
I20260810 14:57:06.811504 17050 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:06.811582 17050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:06.811741 17050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:06.812932 17050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "9adac47064e34c8fb17c730439c34781"
format_stamp: "Formatted at 2026-08-10 14:57:06 on dist-test-slave-0b58"
server_key: "33ba3c07e1c2ad0b4140e5156b4d48c1"
server_key_iv: "0d49532ae13fe4942ac1fd07650bd2c0"
server_key_version: "encryptionkey@0"
I20260810 14:57:06.813315 17050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "9adac47064e34c8fb17c730439c34781"
format_stamp: "Formatted at 2026-08-10 14:57:06 on dist-test-slave-0b58"
server_key: "33ba3c07e1c2ad0b4140e5156b4d48c1"
server_key_iv: "0d49532ae13fe4942ac1fd07650bd2c0"
server_key_version: "encryptionkey@0"
I20260810 14:57:06.814778 17050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:57:06.815614 17065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:06.815812 17050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:06.815932 17050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "9adac47064e34c8fb17c730439c34781"
format_stamp: "Formatted at 2026-08-10 14:57:06 on dist-test-slave-0b58"
server_key: "33ba3c07e1c2ad0b4140e5156b4d48c1"
server_key_iv: "0d49532ae13fe4942ac1fd07650bd2c0"
server_key_version: "encryptionkey@0"
I20260810 14:57:06.816015 17050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:06.825730 17050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:06.829061 17050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:06.829372 17050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:06.829901 17050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:06.830325 17050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:06.830389 17050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:06.830425 17050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:06.830454 17050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:06.837272 17050 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:37651
I20260810 14:57:06.837381 17199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:37651 every 8 connection(s)
I20260810 14:57:06.837764 17050 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
Aug 10 14:57:06 dist-test-slave-0b58 krb5kdc[16929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373826, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:06.847744 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 17050
I20260810 14:57:06.847878 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:57:06.848140 31017 external_mini_cluster.cc:1428] Setting key 1990162dcbe887216b6acf3f416762eb
I20260810 14:57:06.849160 17025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:06.838702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:47175 (local address 127.30.74.126:42263)
0810 14:57:06.838796 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:06.838799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:06.838925 (+ 126us) server_negotiation.cc:407] Connection header received
0810 14:57:06.839460 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:06.839462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:06.839475 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:06.839529 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:06.840734 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:06.841264 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:06.842532 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:06.842786 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:06.842866 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:06.845596 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:06.845598 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:06.845598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:06.845606 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:06.847213 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:06.847779 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:06.847781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:06.847781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:06.847886 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:06.848207 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:06.848209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:06.848209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:06.848399 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:06.848484 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:06.848924 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 14:57:06.849032 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 14:57:06.849355 17202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:06.838598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42263 (local address 127.30.74.65:47175)
0810 14:57:06.838785 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:06.838827 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:06.839357 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:06.839595 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:06.839599 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:06.840317 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:06.840556 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:06.840562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:06.841386 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:06.841387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:06.842408 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:06.842412 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:06.842892 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:06.842893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:06.843025 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:06.843749 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:06.843759 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:06.845461 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:06.847301 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:06.847310 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:06.847313 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:06.847671 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:06.847991 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:06.847992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:06.847993 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:06.848127 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:06.848487 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:06.848496 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:06.848704 (+ 208us) client_negotiation.cc:769] Sending connection context
0810 14:57:06.848833 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 14:57:06.849065 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260810 14:57:06.850109 17200 heartbeater.cc:347] Connected to a master server at 127.30.74.126:42263
I20260810 14:57:06.850263 17200 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:06.850498 17200 heartbeater.cc:511] Master 127.30.74.126:42263 requested a full tablet report, sending...
I20260810 14:57:06.851091 16981 ts_manager.cc:194] Registered new tserver with Master: 9adac47064e34c8fb17c730439c34781 (127.30.74.65:37651)
I20260810 14:57:06.851779 16981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:47175
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:57:06.881948 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42263
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:06.972836 17205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:06.973104 17205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:06.973133 17205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:06.974436 17205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:06.974480 17205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:06.974509 17205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:57:06.976069 17205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:06.976394 17205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:06.976611 17205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:06.982134 17214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:06.982123 17212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:06.982116 17210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:06.982549 17205 server_base.cc:1034] running on GCE node
I20260810 14:57:06.982764 17205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:06.983013 17205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:06.984155 17205 hybrid_clock.cc:648] HybridClock initialized: now 1786373826984120 us; error 51 us; skew 500 ppm
Aug 10 14:57:06 dist-test-slave-0b58 krb5kdc[16929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373826, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:06.986851 17205 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:57:06.987398 17205 webserver.cc:460] Webserver started at http://127.30.74.66:44519/ using document root <none> and password file <none>
I20260810 14:57:06.987636 17205 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:06.987682 17205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:06.987818 17205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:06.989013 17205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "a933bd742d0e4a92a0dbf72ade347af1"
format_stamp: "Formatted at 2026-08-10 14:57:06 on dist-test-slave-0b58"
server_key: "5c205355f7d304b421e42d834030a3a8"
server_key_iv: "2504d686d7bf289cb59eaf17c43ab64a"
server_key_version: "encryptionkey@0"
I20260810 14:57:06.989413 17205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "a933bd742d0e4a92a0dbf72ade347af1"
format_stamp: "Formatted at 2026-08-10 14:57:06 on dist-test-slave-0b58"
server_key: "5c205355f7d304b421e42d834030a3a8"
server_key_iv: "2504d686d7bf289cb59eaf17c43ab64a"
server_key_version: "encryptionkey@0"
I20260810 14:57:06.990861 17205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:57:06.991724 17221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:06.991957 17205 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:06.992084 17205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "a933bd742d0e4a92a0dbf72ade347af1"
format_stamp: "Formatted at 2026-08-10 14:57:06 on dist-test-slave-0b58"
server_key: "5c205355f7d304b421e42d834030a3a8"
server_key_iv: "2504d686d7bf289cb59eaf17c43ab64a"
server_key_version: "encryptionkey@0"
I20260810 14:57:06.992172 17205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:07.004935 17205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:07.007524 17205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:07.007700 17205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:07.007918 17205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:07.008286 17205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:07.008340 17205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:07.008391 17205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:07.008419 17205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:07.014984 17205 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:41493
I20260810 14:57:07.015079 17354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:41493 every 8 connection(s)
I20260810 14:57:07.015465 17205 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:57:07.018664 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 17205
I20260810 14:57:07.018846 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:57:07.019098 31017 external_mini_cluster.cc:1428] Setting key 760a797fddf92e9e0bce07a96a1a8982
Aug 10 14:57:07 dist-test-slave-0b58 krb5kdc[16929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373826, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:07.027776 17025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.016402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:53891 (local address 127.30.74.126:42263)
0810 14:57:07.016469 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.016472 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.016719 (+ 247us) server_negotiation.cc:407] Connection header received
0810 14:57:07.017221 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.017223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.017235 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.017300 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:07.018410 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.019101 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.020423 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.020636 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.020716 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.023580 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:07.023582 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:07.023582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:07.023593 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:07.025668 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:07.026499 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:07.026501 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:07.026502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:07.026583 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:07.026968 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:07.026969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:07.026970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:07.027147 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:07.027231 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.027542 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.027655 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 14:57:07.027920 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.016371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42263 (local address 127.30.74.66:53891)
0810 14:57:07.016558 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:07.016598 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:07.017095 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:07.017427 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:07.017431 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:07.018041 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:07.018254 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.018259 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.019222 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.019224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.020334 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.020337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.020739 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.020740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.020875 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.021684 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:07.021695 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:07.023460 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:07.025833 (+ 2373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:07.025843 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:07.025846 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:07.026394 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:07.026724 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:07.026725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:07.026726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:07.026886 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:07.027228 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:07.027234 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:07.027394 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 14:57:07.027489 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 14:57:07.027660 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":108,"threads_started":1}
I20260810 14:57:07.028560 17355 heartbeater.cc:347] Connected to a master server at 127.30.74.126:42263
I20260810 14:57:07.028671 17355 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:07.028909 17355 heartbeater.cc:511] Master 127.30.74.126:42263 requested a full tablet report, sending...
I20260810 14:57:07.029422 16981 ts_manager.cc:194] Registered new tserver with Master: a933bd742d0e4a92a0dbf72ade347af1 (127.30.74.66:41493)
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
I20260810 14:57:07.029981 16981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:53891
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:57:07.052244 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42263
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:07.140610 17360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:07.140861 17360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:07.140892 17360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:07.142228 17360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:07.142303 17360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:07.142334 17360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:57:07.144141 17360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:42263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:07.144464 17360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:07.144747 17360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:07.151235 17366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:07.151435 17365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:07.151235 17368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:07.151526 17360 server_base.cc:1034] running on GCE node
I20260810 14:57:07.151741 17360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:07.152242 17360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:07.153497 17360 hybrid_clock.cc:648] HybridClock initialized: now 1786373827153478 us; error 49 us; skew 500 ppm
Aug 10 14:57:07 dist-test-slave-0b58 krb5kdc[16929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373827, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:07.156471 17360 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:57:07.157061 17360 webserver.cc:460] Webserver started at http://127.30.74.67:33299/ using document root <none> and password file <none>
I20260810 14:57:07.157344 17360 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:07.157423 17360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:07.157591 17360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:07.158779 17360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "821b085a93b747a8acacc8a119f9e673"
format_stamp: "Formatted at 2026-08-10 14:57:07 on dist-test-slave-0b58"
server_key: "8eb228fe3f5e3cc4fe8078e8511d0ee9"
server_key_iv: "37319e0c00062e9ee408a3d72d6d734e"
server_key_version: "encryptionkey@0"
I20260810 14:57:07.159200 17360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "821b085a93b747a8acacc8a119f9e673"
format_stamp: "Formatted at 2026-08-10 14:57:07 on dist-test-slave-0b58"
server_key: "8eb228fe3f5e3cc4fe8078e8511d0ee9"
server_key_iv: "37319e0c00062e9ee408a3d72d6d734e"
server_key_version: "encryptionkey@0"
I20260810 14:57:07.160924 17360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:57:07.163023 17376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:07.163355 17360 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 14:57:07.163512 17360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "821b085a93b747a8acacc8a119f9e673"
format_stamp: "Formatted at 2026-08-10 14:57:07 on dist-test-slave-0b58"
server_key: "8eb228fe3f5e3cc4fe8078e8511d0ee9"
server_key_iv: "37319e0c00062e9ee408a3d72d6d734e"
server_key_version: "encryptionkey@0"
I20260810 14:57:07.163656 17360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:07.177736 17360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:07.180678 17360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:07.180878 17360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:07.181156 17360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:07.181519 17360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:07.181576 17360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:07.181623 17360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:07.181658 17360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:07.188730 17360 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:38735
I20260810 14:57:07.188774 17507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:38735 every 8 connection(s)
I20260810 14:57:07.189219 17360 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
Aug 10 14:57:07 dist-test-slave-0b58 krb5kdc[16929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373827, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:07.199219 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 17360
I20260810 14:57:07.199332 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:57:07.199559 31017 external_mini_cluster.cc:1428] Setting key a49802d4157416eed4aa52c27b3724c3
I20260810 14:57:07.201146 17025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.190100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:43597 (local address 127.30.74.126:42263)
0810 14:57:07.190180 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.190182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.190441 (+ 259us) server_negotiation.cc:407] Connection header received
0810 14:57:07.190954 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.190955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.190968 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.191038 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:07.192160 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.192786 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.194200 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.194424 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.194503 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.197724 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:07.197727 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:07.197727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:07.197741 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:07.199321 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:07.199921 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:07.199922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:07.199923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:07.199997 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:07.200372 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:07.200373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:07.200373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:07.200537 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:07.200634 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.200955 (+ 321us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.201048 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 14:57:07.201360 17510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.190086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:42263 (local address 127.30.74.67:43597)
0810 14:57:07.190266 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:07.190321 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:07.190839 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:07.191137 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:07.191141 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:07.191803 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:07.192032 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.192038 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.192922 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.192923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.194031 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.194036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.194563 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.194564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.194741 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.195674 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:07.195685 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:07.197584 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:07.199445 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:07.199455 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:07.199457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:07.199838 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:07.200082 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:07.200083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:07.200084 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:07.200264 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:07.200626 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:07.200635 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:07.200801 (+ 166us) client_negotiation.cc:769] Sending connection context
0810 14:57:07.200916 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 14:57:07.201112 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
I20260810 14:57:07.201987 17508 heartbeater.cc:347] Connected to a master server at 127.30.74.126:42263
I20260810 14:57:07.202090 17508 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:07.202328 17508 heartbeater.cc:511] Master 127.30.74.126:42263 requested a full tablet report, sending...
I20260810 14:57:07.202831 16981 ts_manager.cc:194] Registered new tserver with Master: 821b085a93b747a8acacc8a119f9e673 (127.30.74.67:38735)
I20260810 14:57:07.203238 16981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:43597
I20260810 14:57:07.211869 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:57:07.221354 17025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.213587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50488 (local address 127.30.74.126:42263)
0810 14:57:07.213653 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.213655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.213709 (+ 54us) server_negotiation.cc:407] Connection header received
0810 14:57:07.213778 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.213779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.213789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.213868 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:07.214483 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.215007 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.215841 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.216040 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.216110 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.217826 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:07.217827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:07.217827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:07.217838 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:07.219730 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:07.220199 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:07.220200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:07.220201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:07.220516 (+ 315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:07.220765 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:07.220766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:07.220766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:07.220959 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:07.221061 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.221198 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.221281 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 14:57:07.223903 16981 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50488:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:57:07.226013 16981 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:57:07.234277 17522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.229705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:38735 (local address 127.0.0.1:51704)
0810 14:57:07.229925 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:07.229942 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:07.229998 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:07.230405 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:07.230406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:07.230408 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:07.230526 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.230529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.231511 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.231512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.233076 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.233079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.233863 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.233864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.233992 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.234010 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 14:57:07.234113 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 14:57:07.234184 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":25,"thread_start_us":85,"threads_started":1}
I20260810 14:57:07.234287 17523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.229842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51704 (local address 127.30.74.67:38735)
0810 14:57:07.230091 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.230097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.230109 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:57:07.230157 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.230159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.230175 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.230266 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:07.230616 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.231398 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.233173 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.233751 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.233861 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.233940 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.234135 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.234185 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":127,"threads_started":1}
I20260810 14:57:07.234994 17520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.229453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:41493 (local address 127.0.0.1:42068)
0810 14:57:07.229609 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:07.229663 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:07.229754 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:07.230398 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:07.230402 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:07.230405 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:07.230529 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.230532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.232453 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.232453 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.233333 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.233336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.234610 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.234611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.234683 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.234777 (+ 94us) client_negotiation.cc:769] Sending connection context
0810 14:57:07.234820 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 14:57:07.234873 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":14,"thread_start_us":67,"threads_started":1}
I20260810 14:57:07.235178 17431 tablet_service.cc:1467] Processing CreateTablet for tablet a0e6164bfa2c4cfeae2ef66bf7f3f302 (DEFAULT_TABLE table=test-table [id=1da7154dba89401b8d85ff8bcbcef892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:07.235543 17521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.229684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42068 (local address 127.30.74.66:41493)
0810 14:57:07.230123 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.230126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.230135 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:57:07.230234 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.230237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.230253 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.230615 (+ 362us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:07.230696 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.232316 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.233432 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.234322 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.235231 (+ 909us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.235324 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.235356 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.235452 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":100,"threads_started":1}
I20260810 14:57:07.235682 17431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0e6164bfa2c4cfeae2ef66bf7f3f302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:07.236385 17524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.229895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:37651 (local address 127.0.0.1:35466)
0810 14:57:07.230257 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:07.230268 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:07.230312 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:07.230563 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:07.230563 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:07.230565 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:07.230602 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.230603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.232449 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.232450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.234889 (+ 2439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.234892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.236068 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.236069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.236140 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.236146 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:57:07.236225 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 14:57:07.236306 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":3,"thread_start_us":92,"threads_started":1}
I20260810 14:57:07.236409 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.230043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35466 (local address 127.30.74.65:37651)
0810 14:57:07.230287 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.230291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.230303 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:57:07.230383 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.230386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.230401 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.230473 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:07.230680 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.232316 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.235007 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.235955 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.236075 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.236140 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.236244 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.236327 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":107,"threads_started":1}
I20260810 14:57:07.236796 17276 tablet_service.cc:1467] Processing CreateTablet for tablet a0e6164bfa2c4cfeae2ef66bf7f3f302 (DEFAULT_TABLE table=test-table [id=1da7154dba89401b8d85ff8bcbcef892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:07.237176 17276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0e6164bfa2c4cfeae2ef66bf7f3f302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:07.238665 17123 tablet_service.cc:1467] Processing CreateTablet for tablet a0e6164bfa2c4cfeae2ef66bf7f3f302 (DEFAULT_TABLE table=test-table [id=1da7154dba89401b8d85ff8bcbcef892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:07.239087 17123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0e6164bfa2c4cfeae2ef66bf7f3f302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:07.239480 17526 tablet_bootstrap.cc:492] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673: Bootstrap starting.
I20260810 14:57:07.240334 17526 tablet_bootstrap.cc:654] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:07.240863 17527 tablet_bootstrap.cc:492] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1: Bootstrap starting.
I20260810 14:57:07.240955 17526 log.cc:824] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:07.241786 17527 tablet_bootstrap.cc:654] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:07.242127 17526 tablet_bootstrap.cc:492] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673: No bootstrap required, opened a new log
I20260810 14:57:07.242209 17526 ts_tablet_manager.cc:1397] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 14:57:07.242452 17527 log.cc:824] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:07.243111 17530 tablet_bootstrap.cc:492] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781: Bootstrap starting.
I20260810 14:57:07.243445 17527 tablet_bootstrap.cc:492] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1: No bootstrap required, opened a new log
I20260810 14:57:07.243531 17527 ts_tablet_manager.cc:1397] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 14:57:07.243834 17530 tablet_bootstrap.cc:654] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:07.244477 17530 log.cc:824] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:07.244684 17526 raft_consensus.cc:348] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a933bd742d0e4a92a0dbf72ade347af1" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41493 } } peers { permanent_uuid: "821b085a93b747a8acacc8a119f9e673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38735 } } peers { permanent_uuid: "9adac47064e34c8fb17c730439c34781" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37651 } }
I20260810 14:57:07.244923 17526 raft_consensus.cc:374] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:07.244974 17526 raft_consensus.cc:729] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821b085a93b747a8acacc8a119f9e673, State: Initialized, Role: FOLLOWER
I20260810 14:57:07.245146 17526 consensus_queue.cc:260] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a933bd742d0e4a92a0dbf72ade347af1" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41493 } } peers { permanent_uuid: "821b085a93b747a8acacc8a119f9e673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38735 } } peers { permanent_uuid: "9adac47064e34c8fb17c730439c34781" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37651 } }
I20260810 14:57:07.245455 17530 tablet_bootstrap.cc:492] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781: No bootstrap required, opened a new log
I20260810 14:57:07.245491 17526 ts_tablet_manager.cc:1428] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 14:57:07.245532 17530 ts_tablet_manager.cc:1397] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:57:07.245616 17508 heartbeater.cc:503] Master 127.30.74.126:42263 was elected leader, sending a full tablet report...
I20260810 14:57:07.245513 17527 raft_consensus.cc:348] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a933bd742d0e4a92a0dbf72ade347af1" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41493 } } peers { permanent_uuid: "821b085a93b747a8acacc8a119f9e673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38735 } } peers { permanent_uuid: "9adac47064e34c8fb17c730439c34781" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37651 } }
I20260810 14:57:07.245715 17527 raft_consensus.cc:374] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:07.245764 17527 raft_consensus.cc:729] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a933bd742d0e4a92a0dbf72ade347af1, State: Initialized, Role: FOLLOWER
I20260810 14:57:07.246076 17527 consensus_queue.cc:260] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a933bd742d0e4a92a0dbf72ade347af1" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41493 } } peers { permanent_uuid: "821b085a93b747a8acacc8a119f9e673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38735 } } peers { permanent_uuid: "9adac47064e34c8fb17c730439c34781" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37651 } }
I20260810 14:57:07.246405 17527 ts_tablet_manager.cc:1428] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:57:07.246517 17355 heartbeater.cc:503] Master 127.30.74.126:42263 was elected leader, sending a full tablet report...
I20260810 14:57:07.247820 17530 raft_consensus.cc:348] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a933bd742d0e4a92a0dbf72ade347af1" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41493 } } peers { permanent_uuid: "821b085a93b747a8acacc8a119f9e673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38735 } } peers { permanent_uuid: "9adac47064e34c8fb17c730439c34781" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37651 } }
I20260810 14:57:07.247965 17530 raft_consensus.cc:374] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:07.247987 17530 raft_consensus.cc:729] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9adac47064e34c8fb17c730439c34781, State: Initialized, Role: FOLLOWER
I20260810 14:57:07.248098 17530 consensus_queue.cc:260] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a933bd742d0e4a92a0dbf72ade347af1" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41493 } } peers { permanent_uuid: "821b085a93b747a8acacc8a119f9e673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38735 } } peers { permanent_uuid: "9adac47064e34c8fb17c730439c34781" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37651 } }
I20260810 14:57:07.248435 17530 ts_tablet_manager.cc:1428] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260810 14:57:07.248456 17200 heartbeater.cc:503] Master 127.30.74.126:42263 was elected leader, sending a full tablet report...
I20260810 14:57:07.249518 17533 raft_consensus.cc:482] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:57:07.249631 17533 raft_consensus.cc:504] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a933bd742d0e4a92a0dbf72ade347af1" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41493 } } peers { permanent_uuid: "821b085a93b747a8acacc8a119f9e673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38735 } } peers { permanent_uuid: "9adac47064e34c8fb17c730439c34781" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37651 } }
I20260810 14:57:07.250011 17533 leader_election.cc:288] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 821b085a93b747a8acacc8a119f9e673 (127.30.74.67:38735), 9adac47064e34c8fb17c730439c34781 (127.30.74.65:37651)
I20260810 14:57:07.253521 17523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.250319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:52811 (local address 127.30.74.67:38735)
0810 14:57:07.250383 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.250386 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.250396 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:57:07.250426 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.250427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.250459 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.250538 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:07.250788 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.251350 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.252552 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.253088 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.253185 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.253196 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.253381 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.253426 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 14:57:07.253515 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.250184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:38735 (local address 127.30.74.66:52811)
0810 14:57:07.250225 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:07.250238 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:07.250291 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:07.250599 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:07.250601 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:07.250603 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:07.250678 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.250680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.251455 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.251456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.252458 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.252462 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.253194 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.253195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.253298 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.253301 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:57:07.253375 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 14:57:07.253424 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":18}
I20260810 14:57:07.253825 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.250530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:50321 (local address 127.30.74.65:37651)
0810 14:57:07.250569 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.250571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.250580 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:57:07.250644 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.250644 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.250651 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.250697 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:07.250851 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.251335 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.252808 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.253190 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.253466 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.253477 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.253683 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.253744 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 14:57:07.254081 17536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.250324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:37651 (local address 127.30.74.66:50321)
0810 14:57:07.250509 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:07.250522 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:07.250575 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:07.250700 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:07.250701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:07.250702 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:07.250783 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.250784 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.251424 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.251426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.252712 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.252716 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.253465 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.253466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.253561 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.253566 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:57:07.253948 (+ 382us) client_negotiation.cc:240] Negotiation successful
0810 14:57:07.254001 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":28,"thread_start_us":108,"threads_started":1}
I20260810 14:57:07.254094 17454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e6164bfa2c4cfeae2ef66bf7f3f302" candidate_uuid: "a933bd742d0e4a92a0dbf72ade347af1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b085a93b747a8acacc8a119f9e673" is_pre_election: true
I20260810 14:57:07.254280 17454 raft_consensus.cc:2454] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a933bd742d0e4a92a0dbf72ade347af1 in term 0.
I20260810 14:57:07.254527 17222 leader_election.cc:302] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b085a93b747a8acacc8a119f9e673, a933bd742d0e4a92a0dbf72ade347af1; no voters:
I20260810 14:57:07.254524 17147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e6164bfa2c4cfeae2ef66bf7f3f302" candidate_uuid: "a933bd742d0e4a92a0dbf72ade347af1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9adac47064e34c8fb17c730439c34781" is_pre_election: true
I20260810 14:57:07.254659 17533 raft_consensus.cc:2781] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:57:07.254711 17533 raft_consensus.cc:482] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:57:07.254729 17147 raft_consensus.cc:2454] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a933bd742d0e4a92a0dbf72ade347af1 in term 0.
I20260810 14:57:07.254760 17533 raft_consensus.cc:3037] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:07.255541 17533 raft_consensus.cc:504] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a933bd742d0e4a92a0dbf72ade347af1" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41493 } } peers { permanent_uuid: "821b085a93b747a8acacc8a119f9e673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38735 } } peers { permanent_uuid: "9adac47064e34c8fb17c730439c34781" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37651 } }
I20260810 14:57:07.255695 17533 leader_election.cc:288] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [CANDIDATE]: Term 1 election: Requested vote from peers 821b085a93b747a8acacc8a119f9e673 (127.30.74.67:38735), 9adac47064e34c8fb17c730439c34781 (127.30.74.65:37651)
I20260810 14:57:07.255898 17454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e6164bfa2c4cfeae2ef66bf7f3f302" candidate_uuid: "a933bd742d0e4a92a0dbf72ade347af1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "821b085a93b747a8acacc8a119f9e673"
I20260810 14:57:07.255908 17147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0e6164bfa2c4cfeae2ef66bf7f3f302" candidate_uuid: "a933bd742d0e4a92a0dbf72ade347af1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9adac47064e34c8fb17c730439c34781"
I20260810 14:57:07.255980 17454 raft_consensus.cc:3037] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:07.256011 17147 raft_consensus.cc:3037] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:07.256857 17454 raft_consensus.cc:2454] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a933bd742d0e4a92a0dbf72ade347af1 in term 1.
I20260810 14:57:07.257026 17222 leader_election.cc:302] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 821b085a93b747a8acacc8a119f9e673, a933bd742d0e4a92a0dbf72ade347af1; no voters:
I20260810 14:57:07.257133 17533 raft_consensus.cc:2781] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:07.257248 17147 raft_consensus.cc:2454] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a933bd742d0e4a92a0dbf72ade347af1 in term 1.
I20260810 14:57:07.257354 17533 raft_consensus.cc:686] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [term 1 LEADER]: Becoming Leader. State: Replica: a933bd742d0e4a92a0dbf72ade347af1, State: Running, Role: LEADER
I20260810 14:57:07.257455 17533 consensus_queue.cc:237] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a933bd742d0e4a92a0dbf72ade347af1" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41493 } } peers { permanent_uuid: "821b085a93b747a8acacc8a119f9e673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38735 } } peers { permanent_uuid: "9adac47064e34c8fb17c730439c34781" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37651 } }
I20260810 14:57:07.258504 16981 catalog_manager.cc:5495] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a933bd742d0e4a92a0dbf72ade347af1 (127.30.74.66). New cstate: current_term: 1 leader_uuid: "a933bd742d0e4a92a0dbf72ade347af1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a933bd742d0e4a92a0dbf72ade347af1" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 41493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "821b085a93b747a8acacc8a119f9e673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9adac47064e34c8fb17c730439c34781" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37651 } health_report { overall_health: UNKNOWN } } }
W20260810 14:57:07.266077 17356 tablet.cc:2369] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:57:07.271106 17521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.268880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42078 (local address 127.30.74.66:41493)
0810 14:57:07.268948 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.268950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.268958 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:57:07.268990 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.268991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.269008 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.269071 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:07.269162 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.269621 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.270417 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.270638 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.270706 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.270837 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:07.270937 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:07.270996 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.271025 (+ 29us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.271043 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 14:57:07.273108 17147 raft_consensus.cc:1261] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781 [term 1 FOLLOWER]: Refusing update from remote peer a933bd742d0e4a92a0dbf72ade347af1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:57:07.273124 17454 raft_consensus.cc:1261] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673 [term 1 FOLLOWER]: Refusing update from remote peer a933bd742d0e4a92a0dbf72ade347af1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:57:07.273543 17537 consensus_queue.cc:1035] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "821b085a93b747a8acacc8a119f9e673" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:57:07.273686 17533 consensus_queue.cc:1035] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9adac47064e34c8fb17c730439c34781" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:57:07.276599 17540 mvcc.cc:205] Tried to move back new op lower bound from 7316987196507852800 to 7316987196447514624. Current Snapshot: MvccSnapshot[applied={T|T < 7316987196507852800}]
I20260810 14:57:07.279158 16981 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:57:07.279269 16981 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:57:07.280084 16981 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b36b6d3b3ccf442395a589f1113af5cb: Sending DeleteTablet for 3 replicas of tablet a0e6164bfa2c4cfeae2ef66bf7f3f302
I20260810 14:57:07.280486 17276 tablet_service.cc:1514] Processing DeleteTablet for tablet a0e6164bfa2c4cfeae2ef66bf7f3f302 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:07 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:42068
I20260810 14:57:07.280599 17123 tablet_service.cc:1514] Processing DeleteTablet for tablet a0e6164bfa2c4cfeae2ef66bf7f3f302 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:07 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:35466
I20260810 14:57:07.280747 17431 tablet_service.cc:1514] Processing DeleteTablet for tablet a0e6164bfa2c4cfeae2ef66bf7f3f302 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:07 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:51704
I20260810 14:57:07.280854 17551 tablet_replica.cc:330] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781: stopping tablet replica
I20260810 14:57:07.280956 17551 raft_consensus.cc:2229] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:57:07.280977 17552 tablet_replica.cc:330] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673: stopping tablet replica
I20260810 14:57:07.281057 17551 raft_consensus.cc:2258] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 9adac47064e34c8fb17c730439c34781 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:57:07.281087 17552 raft_consensus.cc:2229] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:57:07.281231 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 17050
I20260810 14:57:07.281273 17552 raft_consensus.cc:2258] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:57:07.283123 17552 ts_tablet_manager.cc:1905] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:57:07.285748 17552 ts_tablet_manager.cc:1918] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:57:07.285851 17552 log.cc:1196] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/a0e6164bfa2c4cfeae2ef66bf7f3f302
I20260810 14:57:07.286274 17552 ts_tablet_manager.cc:1939] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P 821b085a93b747a8acacc8a119f9e673: Deleting consensus metadata
I20260810 14:57:07.286907 16963 catalog_manager.cc:4841] TS 821b085a93b747a8acacc8a119f9e673 (127.30.74.67:38735): tablet a0e6164bfa2c4cfeae2ef66bf7f3f302 (table test-table [id=1da7154dba89401b8d85ff8bcbcef892]) successfully deleted
I20260810 14:57:07.289220 17550 tablet_replica.cc:330] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1: stopping tablet replica
I20260810 14:57:07.289368 17550 raft_consensus.cc:2229] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:57:07.289503 17550 raft_consensus.cc:2258] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:57:07.290297 17550 ts_tablet_manager.cc:1905] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:57:07.292243 17550 ts_tablet_manager.cc:1918] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:57:07.292359 17550 log.cc:1196] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/a0e6164bfa2c4cfeae2ef66bf7f3f302
I20260810 14:57:07.292675 17550 ts_tablet_manager.cc:1939] T a0e6164bfa2c4cfeae2ef66bf7f3f302 P a933bd742d0e4a92a0dbf72ade347af1: Deleting consensus metadata
I20260810 14:57:07.293200 16963 catalog_manager.cc:4841] TS a933bd742d0e4a92a0dbf72ade347af1 (127.30.74.66:41493): tablet a0e6164bfa2c4cfeae2ef66bf7f3f302 (table test-table [id=1da7154dba89401b8d85ff8bcbcef892]) successfully deleted
W20260810 14:57:07.294755 16963 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:37651 (error 108)
I20260810 14:57:07.295055 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 17205
I20260810 14:57:07.296169 17524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.295179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:37651 (local address 127.0.0.1:35476)
0810 14:57:07.296074 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:07.296120 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":872}
W20260810 14:57:07.296435 16963 catalog_manager.cc:4584] TS 9adac47064e34c8fb17c730439c34781 (127.30.74.65:37651): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a0e6164bfa2c4cfeae2ef66bf7f3f302: Network error: Client connection negotiation failed: client connection to 127.30.74.65:37651: connect: Connection refused (error 111)
I20260810 14:57:07.302167 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 17360
I20260810 14:57:07.310698 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 16944
2026-08-10T14:57:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2157 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T14:57:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:07Z Disabled control of system clock
I20260810 14:57:07.346148 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:41513
--webserver_interface=127.30.74.126
--webserver_port=0
--builtin_ntp_servers=127.30.74.84:35689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:41513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:57:07.437439 17557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:07.437708 17557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:07.437733 17557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:07.439152 17557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:07.439217 17557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:07.439231 17557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:07.439244 17557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:07.439257 17557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:07.441615 17557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:41513
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:41513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:07.441958 17557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:07.442320 17557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:07.447755 17564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:07.447769 17563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:07.447877 17566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:07.448094 17557 server_base.cc:1034] running on GCE node
I20260810 14:57:07.448421 17557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:07.448763 17557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:07.449941 17557 hybrid_clock.cc:648] HybridClock initialized: now 1786373827449937 us; error 40 us; skew 500 ppm
I20260810 14:57:07.451280 17557 webserver.cc:460] Webserver started at http://127.30.74.126:45835/ using document root <none> and password file <none>
I20260810 14:57:07.451555 17557 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:07.451627 17557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:07.451792 17557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:07.453003 17557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "9fcdb7e151684f0891f2f47460b6348c"
format_stamp: "Formatted at 2026-08-10 14:57:07 on dist-test-slave-0b58"
server_key: "f74645bf07734a110c6a58ac14b9cd52"
server_key_iv: "7df25705ee5e7d1bd94fe0db08d9027d"
server_key_version: "encryptionkey@0"
I20260810 14:57:07.453404 17557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "9fcdb7e151684f0891f2f47460b6348c"
format_stamp: "Formatted at 2026-08-10 14:57:07 on dist-test-slave-0b58"
server_key: "f74645bf07734a110c6a58ac14b9cd52"
server_key_iv: "7df25705ee5e7d1bd94fe0db08d9027d"
server_key_version: "encryptionkey@0"
I20260810 14:57:07.454984 17557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:57:07.455928 17574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:07.456171 17557 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:07.456315 17557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "9fcdb7e151684f0891f2f47460b6348c"
format_stamp: "Formatted at 2026-08-10 14:57:07 on dist-test-slave-0b58"
server_key: "f74645bf07734a110c6a58ac14b9cd52"
server_key_iv: "7df25705ee5e7d1bd94fe0db08d9027d"
server_key_version: "encryptionkey@0"
I20260810 14:57:07.456416 17557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:07.464038 17557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:07.464465 17557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:07.464636 17557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:07.468951 17557 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:41513
I20260810 14:57:07.469008 17634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:41513 every 8 connection(s)
I20260810 14:57:07.469383 17557 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:07.470449 17635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:07.472281 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 17557
I20260810 14:57:07.472415 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:07.472632 31017 external_mini_cluster.cc:1428] Setting key dd6c6f952d59603b264072863e93e778
I20260810 14:57:07.473313 17635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c: Bootstrap starting.
I20260810 14:57:07.473928 17635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:07.474241 17635 log.cc:824] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:07.475076 17635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c: No bootstrap required, opened a new log
I20260810 14:57:07.477424 17635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcdb7e151684f0891f2f47460b6348c" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41513 } }
I20260810 14:57:07.477625 17635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:07.477663 17635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fcdb7e151684f0891f2f47460b6348c, State: Initialized, Role: FOLLOWER
I20260810 14:57:07.477820 17635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcdb7e151684f0891f2f47460b6348c" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41513 } }
I20260810 14:57:07.477921 17635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:07.477955 17635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:07.478003 17635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:07.478909 17635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcdb7e151684f0891f2f47460b6348c" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41513 } }
I20260810 14:57:07.479053 17635 leader_election.cc:302] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fcdb7e151684f0891f2f47460b6348c; no voters:
I20260810 14:57:07.479156 17638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.473220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51478 (local address 127.30.74.126:41513)
0810 14:57:07.473447 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.473457 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.473492 (+ 35us) server_negotiation.cc:407] Connection header received
0810 14:57:07.473926 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.473932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.473996 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.474173 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:07.474430 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.475840 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.476730 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.477443 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.477567 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.478357 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:07.478362 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:07.478362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:07.478593 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:07.478641 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:07.478656 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:07.478731 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.478799 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.478888 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260810 14:57:07.479254 17635 leader_election.cc:288] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:07.479408 17640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:07.479576 17640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [term 1 LEADER]: Becoming Leader. State: Replica: 9fcdb7e151684f0891f2f47460b6348c, State: Running, Role: LEADER
I20260810 14:57:07.479671 17640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcdb7e151684f0891f2f47460b6348c" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41513 } }
I20260810 14:57:07.479969 17635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:07.480074 17641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fcdb7e151684f0891f2f47460b6348c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcdb7e151684f0891f2f47460b6348c" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41513 } } }
I20260810 14:57:07.480087 17643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fcdb7e151684f0891f2f47460b6348c. Latest consensus state: current_term: 1 leader_uuid: "9fcdb7e151684f0891f2f47460b6348c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fcdb7e151684f0891f2f47460b6348c" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41513 } } }
I20260810 14:57:07.480170 17641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:07.480186 17643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:07.480458 17645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:07.481077 17645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:07.483104 17645 catalog_manager.cc:1347] Generated new cluster ID: 435451ab18974812a7a7fc2a42abaaf8
I20260810 14:57:07.483174 17645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:07.490746 17645 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:07.491523 17645 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:07.496915 17645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c: Generated new TSK 0
I20260810 14:57:07.497159 17645 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:07.500501 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:41513
--builtin_ntp_servers=127.30.74.84:35689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:57:07.592789 17662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:07.593024 17662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:07.593046 17662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:07.594345 17662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:07.594399 17662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:07.594427 17662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:57:07.595888 17662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:41513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:07.596175 17662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:07.596537 17662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:07.602700 17671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:07.602712 17668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:07.602723 17669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:07.603034 17662 server_base.cc:1034] running on GCE node
I20260810 14:57:07.603248 17662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:07.603524 17662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:07.604707 17662 hybrid_clock.cc:648] HybridClock initialized: now 1786373827604677 us; error 48 us; skew 500 ppm
I20260810 14:57:07.606122 17662 webserver.cc:460] Webserver started at http://127.30.74.65:39027/ using document root <none> and password file <none>
I20260810 14:57:07.606416 17662 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:07.606495 17662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:07.606678 17662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:07.607893 17662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "f7690ee176c94217927448be1f56d784"
format_stamp: "Formatted at 2026-08-10 14:57:07 on dist-test-slave-0b58"
server_key: "039fd84acb6878909daf54242bf73040"
server_key_iv: "e45eb1c3c372315f8c0de025381ce8ba"
server_key_version: "encryptionkey@0"
I20260810 14:57:07.608319 17662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "f7690ee176c94217927448be1f56d784"
format_stamp: "Formatted at 2026-08-10 14:57:07 on dist-test-slave-0b58"
server_key: "039fd84acb6878909daf54242bf73040"
server_key_iv: "e45eb1c3c372315f8c0de025381ce8ba"
server_key_version: "encryptionkey@0"
I20260810 14:57:07.609849 17662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:57:07.610764 17678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:07.610971 17662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:57:07.611083 17662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "f7690ee176c94217927448be1f56d784"
format_stamp: "Formatted at 2026-08-10 14:57:07 on dist-test-slave-0b58"
server_key: "039fd84acb6878909daf54242bf73040"
server_key_iv: "e45eb1c3c372315f8c0de025381ce8ba"
server_key_version: "encryptionkey@0"
I20260810 14:57:07.611174 17662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:07.625447 17662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:07.625850 17662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:07.626027 17662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:07.626312 17662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:07.626710 17662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:07.626777 17662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:07.626814 17662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:07.626860 17662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:07.633590 17662 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:38755
I20260810 14:57:07.633633 17809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:38755 every 8 connection(s)
I20260810 14:57:07.634044 17662 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:57:07.638065 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 17662
I20260810 14:57:07.638176 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:57:07.638453 31017 external_mini_cluster.cc:1428] Setting key 29b5f260e14252bab7857e0e01dd1a6a
I20260810 14:57:07.639633 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:41513
--builtin_ntp_servers=127.30.74.84:35689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:57:07.640249 17638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.634946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:49157 (local address 127.30.74.126:41513)
0810 14:57:07.635017 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.635019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.635336 (+ 317us) server_negotiation.cc:407] Connection header received
0810 14:57:07.635902 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.635904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.635921 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.635989 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:07.636769 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.637288 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.638400 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.638695 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.638787 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.639920 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:07.639921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:07.639922 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:07.639927 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:07.639947 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:07.639952 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:07.640015 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.640097 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.640127 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 14:57:07.640529 17812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.634937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:41513 (local address 127.30.74.65:49157)
0810 14:57:07.635153 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:07.635198 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:07.635786 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:07.636045 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:07.636049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:07.636057 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:07.636676 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.636683 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.637406 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.637407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.638300 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.638304 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.638808 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.638809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.639003 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.639722 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:57:07.639722 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:07.639777 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:57:07.639818 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:07.640018 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:07.640021 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:07.640024 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:57:07.640078 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 14:57:07.640200 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":100,"threads_started":1}
I20260810 14:57:07.641824 17810 heartbeater.cc:347] Connected to a master server at 127.30.74.126:41513
I20260810 14:57:07.641952 17810 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:07.642175 17810 heartbeater.cc:511] Master 127.30.74.126:41513 requested a full tablet report, sending...
I20260810 14:57:07.642747 17592 ts_manager.cc:194] Registered new tserver with Master: f7690ee176c94217927448be1f56d784 (127.30.74.65:38755)
I20260810 14:57:07.643405 17592 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.74.65:49157
W20260810 14:57:07.727178 17813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:07.727489 17813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:07.727535 17813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:07.729007 17813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:07.729099 17813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:07.729144 17813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:57:07.730808 17813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:41513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:07.731375 17813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:07.731849 17813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:07.738286 17823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:07.738319 17818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:07.738270 17819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:07.738452 17813 server_base.cc:1034] running on GCE node
I20260810 14:57:07.738806 17813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:07.739184 17813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:07.740387 17813 hybrid_clock.cc:648] HybridClock initialized: now 1786373827740365 us; error 49 us; skew 500 ppm
I20260810 14:57:07.741967 17813 webserver.cc:460] Webserver started at http://127.30.74.66:46333/ using document root <none> and password file <none>
I20260810 14:57:07.742267 17813 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:07.742355 17813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:07.742524 17813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:07.743707 17813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "c2d2622adc8b4c89b890521abe858a88"
format_stamp: "Formatted at 2026-08-10 14:57:07 on dist-test-slave-0b58"
server_key: "f9f5c4881875b08e701783bd4a6f7736"
server_key_iv: "0123d3b3e1634b385d09cb55d8f24446"
server_key_version: "encryptionkey@0"
I20260810 14:57:07.744120 17813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "c2d2622adc8b4c89b890521abe858a88"
format_stamp: "Formatted at 2026-08-10 14:57:07 on dist-test-slave-0b58"
server_key: "f9f5c4881875b08e701783bd4a6f7736"
server_key_iv: "0123d3b3e1634b385d09cb55d8f24446"
server_key_version: "encryptionkey@0"
I20260810 14:57:07.745647 17813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:57:07.746606 17830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:07.746855 17813 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:57:07.746971 17813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "c2d2622adc8b4c89b890521abe858a88"
format_stamp: "Formatted at 2026-08-10 14:57:07 on dist-test-slave-0b58"
server_key: "f9f5c4881875b08e701783bd4a6f7736"
server_key_iv: "0123d3b3e1634b385d09cb55d8f24446"
server_key_version: "encryptionkey@0"
I20260810 14:57:07.747059 17813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:07.763592 17813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:07.764001 17813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:07.764190 17813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:07.764509 17813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:07.764922 17813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:07.764986 17813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:07.765033 17813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:07.765081 17813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:07.772217 17813 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:45795
I20260810 14:57:07.772344 17961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:45795 every 8 connection(s)
I20260810 14:57:07.772768 17813 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:57:07.776726 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 17813
I20260810 14:57:07.776850 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:57:07.777047 31017 external_mini_cluster.cc:1428] Setting key d3dfeea2325f9aa45a3da99760455d1c
I20260810 14:57:07.778193 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:41513
--builtin_ntp_servers=127.30.74.84:35689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:57:07.779163 17638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.773762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:57657 (local address 127.30.74.126:41513)
0810 14:57:07.773838 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.773840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.774108 (+ 268us) server_negotiation.cc:407] Connection header received
0810 14:57:07.774683 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.774685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.774698 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.774752 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:07.775537 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.776053 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.777469 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.777678 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.777765 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.778851 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:07.778852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:07.778853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:07.778858 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:07.778877 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:07.778882 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:07.778936 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.779046 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.779073 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 14:57:07.779503 17964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.773734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:41513 (local address 127.30.74.66:57657)
0810 14:57:07.773966 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:07.774004 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:07.774546 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:07.774859 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:07.774862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:07.774871 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:07.775416 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.775420 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.776158 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.776159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.777367 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.777371 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.777777 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.777778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.777956 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.778663 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:57:07.778664 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:07.778706 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:57:07.778745 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:07.778954 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:07.778961 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:07.778964 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:57:07.779041 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 14:57:07.779163 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":102,"threads_started":1}
I20260810 14:57:07.780125 17962 heartbeater.cc:347] Connected to a master server at 127.30.74.126:41513
I20260810 14:57:07.780355 17962 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:07.780615 17962 heartbeater.cc:511] Master 127.30.74.126:41513 requested a full tablet report, sending...
I20260810 14:57:07.781077 17592 ts_manager.cc:194] Registered new tserver with Master: c2d2622adc8b4c89b890521abe858a88 (127.30.74.66:45795)
I20260810 14:57:07.781539 17592 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.74.66:57657
W20260810 14:57:07.866219 17965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:07.866464 17965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:07.866489 17965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:07.867789 17965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:07.867843 17965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:07.867871 17965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:57:07.869371 17965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:41513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:07.869664 17965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:07.870015 17965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:07.875960 17972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:07.875973 17971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:07.876375 17965 server_base.cc:1034] running on GCE node
W20260810 14:57:07.876500 17974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:07.876823 17965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:07.877182 17965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:07.878381 17965 hybrid_clock.cc:648] HybridClock initialized: now 1786373827878371 us; error 36 us; skew 500 ppm
I20260810 14:57:07.880154 17965 webserver.cc:460] Webserver started at http://127.30.74.67:33063/ using document root <none> and password file <none>
I20260810 14:57:07.880496 17965 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:07.880601 17965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:07.880792 17965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:07.882148 17965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "3369de05d48c460a8f315bd3cbf0403b"
format_stamp: "Formatted at 2026-08-10 14:57:07 on dist-test-slave-0b58"
server_key: "9050a84fc799ff0616c92df57dd7744d"
server_key_iv: "566e0a2719180e868c30c51aa360e9e9"
server_key_version: "encryptionkey@0"
I20260810 14:57:07.882580 17965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "3369de05d48c460a8f315bd3cbf0403b"
format_stamp: "Formatted at 2026-08-10 14:57:07 on dist-test-slave-0b58"
server_key: "9050a84fc799ff0616c92df57dd7744d"
server_key_iv: "566e0a2719180e868c30c51aa360e9e9"
server_key_version: "encryptionkey@0"
I20260810 14:57:07.884207 17965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260810 14:57:07.885417 17981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:07.885694 17965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:07.885815 17965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "3369de05d48c460a8f315bd3cbf0403b"
format_stamp: "Formatted at 2026-08-10 14:57:07 on dist-test-slave-0b58"
server_key: "9050a84fc799ff0616c92df57dd7744d"
server_key_iv: "566e0a2719180e868c30c51aa360e9e9"
server_key_version: "encryptionkey@0"
I20260810 14:57:07.885903 17965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:07.894979 17965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:07.895390 17965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:07.895561 17965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:07.895859 17965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:07.896333 17965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:07.896406 17965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:07.896456 17965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:07.896486 17965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:07.903337 17965 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:35097
I20260810 14:57:07.903386 18115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:35097 every 8 connection(s)
I20260810 14:57:07.903771 17965 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:57:07.905157 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 17965
I20260810 14:57:07.905269 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:57:07.905499 31017 external_mini_cluster.cc:1428] Setting key ba7a8265edb3d52c3ce307df57fd5e67
I20260810 14:57:07.909958 17638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.904630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:38001 (local address 127.30.74.126:41513)
0810 14:57:07.904706 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.904709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.904972 (+ 263us) server_negotiation.cc:407] Connection header received
0810 14:57:07.905773 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.905775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.905789 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.905861 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:07.906847 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.907348 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.908503 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.908696 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.908756 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.909680 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:07.909681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:07.909681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:07.909687 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:07.909703 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:07.909707 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:07.909764 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.909866 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.909892 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 14:57:07.910180 18118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.904602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:41513 (local address 127.30.74.67:38001)
0810 14:57:07.904814 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:07.904865 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:07.905640 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:07.905983 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:07.905988 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:07.905999 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:07.906751 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.906757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.907459 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.907460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.908418 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.908421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.908794 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.908795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.908915 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.909504 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:57:07.909504 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:07.909536 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:57:07.909562 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:07.909781 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:07.909786 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:07.909789 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:57:07.909850 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 14:57:07.909945 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260810 14:57:07.910751 18116 heartbeater.cc:347] Connected to a master server at 127.30.74.126:41513
I20260810 14:57:07.910858 18116 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:07.911026 18116 heartbeater.cc:511] Master 127.30.74.126:41513 requested a full tablet report, sending...
I20260810 14:57:07.911414 17592 ts_manager.cc:194] Registered new tserver with Master: 3369de05d48c460a8f315bd3cbf0403b (127.30.74.67:35097)
I20260810 14:57:07.911795 17592 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.74.67:38001
I20260810 14:57:07.918118 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:57:07.922899 17638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.919782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51490 (local address 127.30.74.126:41513)
0810 14:57:07.919869 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.919871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.920019 (+ 148us) server_negotiation.cc:407] Connection header received
0810 14:57:07.920051 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.920052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.920063 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.920124 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:07.920386 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.920887 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.921650 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.921844 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.921903 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.922657 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:07.922658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:07.922658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:07.922663 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:07.922679 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:07.922683 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:07.922733 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.922795 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.922824 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 14:57:07.925176 17592 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:57:07.927273 17592 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:57:07.934684 18129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.930372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59284 (local address 127.30.74.65:38755)
0810 14:57:07.930579 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.930585 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.930596 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:57:07.930642 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.930645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.930662 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.930740 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:07.930935 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.932115 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.933539 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.934199 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.934280 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.934379 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.934551 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.934593 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":101,"threads_started":1}
I20260810 14:57:07.934712 18125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.930079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:38755 (local address 127.0.0.1:59284)
0810 14:57:07.930261 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:07.930292 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:07.930382 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:07.930764 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:07.930767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:07.930769 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:07.930848 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.930850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.932206 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.932207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.933458 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.933460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.934310 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.934312 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.934425 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.934442 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 14:57:07.934523 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 14:57:07.934602 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":54,"thread_start_us":63,"threads_started":1}
I20260810 14:57:07.934712 18126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.930087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:35097 (local address 127.0.0.1:38562)
0810 14:57:07.930860 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:07.930869 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:07.930914 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:07.931171 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:07.931172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:07.931174 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:07.931222 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.931223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.932488 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.932489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.933350 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.933353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.934325 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.934326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.934425 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.934442 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 14:57:07.934523 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 14:57:07.934602 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"mutex_wait_us":33,"thread_start_us":124,"threads_started":1}
I20260810 14:57:07.935247 18130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.930412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38562 (local address 127.30.74.67:35097)
0810 14:57:07.930663 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.930667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.930973 (+ 306us) server_negotiation.cc:407] Connection header received
0810 14:57:07.931018 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.931020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.931037 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.931103 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:07.931327 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.932405 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.933449 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.934199 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.934280 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.934380 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.935078 (+ 698us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.935163 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":139,"threads_started":1}
I20260810 14:57:07.935665 17732 tablet_service.cc:1467] Processing CreateTablet for tablet ff845c95a88b4a5ea78b5ff42347ae7d (DEFAULT_TABLE table=test-table [id=a5c1cb6f331642f197eea44941039cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:07.935945 18127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.930079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:45795 (local address 127.0.0.1:57158)
0810 14:57:07.930699 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:07.930709 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:07.930765 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:07.931066 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:07.931067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:07.931069 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:07.931110 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.931112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.932026 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.932027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.933458 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:07.933460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.935685 (+ 2225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.935687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:07.935736 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.935739 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:57:07.935822 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 14:57:07.935869 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":11,"thread_start_us":98,"threads_started":1}
I20260810 14:57:07.935962 18128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:07.930320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57158 (local address 127.30.74.66:45795)
0810 14:57:07.930594 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:07.930600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:07.930900 (+ 300us) server_negotiation.cc:407] Connection header received
0810 14:57:07.930949 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:07.930952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:07.930969 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:07.931052 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:07.931177 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.931932 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.934423 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:07.935501 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:07.935573 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:07.935653 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:07.935840 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 14:57:07.935880 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":90,"threads_started":1}
I20260810 14:57:07.935911 18040 tablet_service.cc:1467] Processing CreateTablet for tablet ff845c95a88b4a5ea78b5ff42347ae7d (DEFAULT_TABLE table=test-table [id=a5c1cb6f331642f197eea44941039cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:07.936096 17732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff845c95a88b4a5ea78b5ff42347ae7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:07.936213 18040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff845c95a88b4a5ea78b5ff42347ae7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:07.936604 17887 tablet_service.cc:1467] Processing CreateTablet for tablet ff845c95a88b4a5ea78b5ff42347ae7d (DEFAULT_TABLE table=test-table [id=a5c1cb6f331642f197eea44941039cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:07.936846 17887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff845c95a88b4a5ea78b5ff42347ae7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:07.939432 18131 tablet_bootstrap.cc:492] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88: Bootstrap starting.
I20260810 14:57:07.939980 18132 tablet_bootstrap.cc:492] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b: Bootstrap starting.
I20260810 14:57:07.940320 18131 tablet_bootstrap.cc:654] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:07.940416 18133 tablet_bootstrap.cc:492] T ff845c95a88b4a5ea78b5ff42347ae7d P f7690ee176c94217927448be1f56d784: Bootstrap starting.
I20260810 14:57:07.940658 18131 log.cc:824] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:07.940721 18132 tablet_bootstrap.cc:654] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:07.941051 18132 log.cc:824] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:07.941147 18133 tablet_bootstrap.cc:654] T ff845c95a88b4a5ea78b5ff42347ae7d P f7690ee176c94217927448be1f56d784: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:07.941493 18133 log.cc:824] T ff845c95a88b4a5ea78b5ff42347ae7d P f7690ee176c94217927448be1f56d784: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:07.941630 18131 tablet_bootstrap.cc:492] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88: No bootstrap required, opened a new log
I20260810 14:57:07.941716 18131 ts_tablet_manager.cc:1397] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 14:57:07.941854 18132 tablet_bootstrap.cc:492] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b: No bootstrap required, opened a new log
I20260810 14:57:07.941936 18132 ts_tablet_manager.cc:1397] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:57:07.942309 18133 tablet_bootstrap.cc:492] T ff845c95a88b4a5ea78b5ff42347ae7d P f7690ee176c94217927448be1f56d784: No bootstrap required, opened a new log
I20260810 14:57:07.942382 18133 ts_tablet_manager.cc:1397] T ff845c95a88b4a5ea78b5ff42347ae7d P f7690ee176c94217927448be1f56d784: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:57:07.944412 18131 raft_consensus.cc:348] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7690ee176c94217927448be1f56d784" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38755 } } peers { permanent_uuid: "c2d2622adc8b4c89b890521abe858a88" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 45795 } } peers { permanent_uuid: "3369de05d48c460a8f315bd3cbf0403b" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35097 } }
I20260810 14:57:07.944638 18131 raft_consensus.cc:374] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:07.944686 18131 raft_consensus.cc:729] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2d2622adc8b4c89b890521abe858a88, State: Initialized, Role: FOLLOWER
I20260810 14:57:07.944836 18131 consensus_queue.cc:260] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7690ee176c94217927448be1f56d784" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38755 } } peers { permanent_uuid: "c2d2622adc8b4c89b890521abe858a88" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 45795 } } peers { permanent_uuid: "3369de05d48c460a8f315bd3cbf0403b" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35097 } }
I20260810 14:57:07.944875 18132 raft_consensus.cc:348] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7690ee176c94217927448be1f56d784" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38755 } } peers { permanent_uuid: "c2d2622adc8b4c89b890521abe858a88" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 45795 } } peers { permanent_uuid: "3369de05d48c460a8f315bd3cbf0403b" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35097 } }
I20260810 14:57:07.945058 18132 raft_consensus.cc:374] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:07.945096 18132 raft_consensus.cc:729] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3369de05d48c460a8f315bd3cbf0403b, State: Initialized, Role: FOLLOWER
I20260810 14:57:07.945161 18131 ts_tablet_manager.cc:1428] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 14:57:07.945297 17962 heartbeater.cc:503] Master 127.30.74.126:41513 was elected leader, sending a full tablet report...
I20260810 14:57:07.945235 18132 consensus_queue.cc:260] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7690ee176c94217927448be1f56d784" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38755 } } peers { permanent_uuid: "c2d2622adc8b4c89b890521abe858a88" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 45795 } } peers { permanent_uuid: "3369de05d48c460a8f315bd3cbf0403b" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35097 } }
I20260810 14:57:07.945258 18133 raft_consensus.cc:348] T ff845c95a88b4a5ea78b5ff42347ae7d P f7690ee176c94217927448be1f56d784 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7690ee176c94217927448be1f56d784" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38755 } } peers { permanent_uuid: "c2d2622adc8b4c89b890521abe858a88" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 45795 } } peers { permanent_uuid: "3369de05d48c460a8f315bd3cbf0403b" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35097 } }
I20260810 14:57:07.945420 18133 raft_consensus.cc:374] T ff845c95a88b4a5ea78b5ff42347ae7d P f7690ee176c94217927448be1f56d784 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:07.945467 18133 raft_consensus.cc:729] T ff845c95a88b4a5ea78b5ff42347ae7d P f7690ee176c94217927448be1f56d784 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7690ee176c94217927448be1f56d784, State: Initialized, Role: FOLLOWER
I20260810 14:57:07.945535 18132 ts_tablet_manager.cc:1428] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260810 14:57:07.945652 18116 heartbeater.cc:503] Master 127.30.74.126:41513 was elected leader, sending a full tablet report...
I20260810 14:57:07.945603 18133 consensus_queue.cc:260] T ff845c95a88b4a5ea78b5ff42347ae7d P f7690ee176c94217927448be1f56d784 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7690ee176c94217927448be1f56d784" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38755 } } peers { permanent_uuid: "c2d2622adc8b4c89b890521abe858a88" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 45795 } } peers { permanent_uuid: "3369de05d48c460a8f315bd3cbf0403b" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35097 } }
I20260810 14:57:07.946203 18133 ts_tablet_manager.cc:1428] T ff845c95a88b4a5ea78b5ff42347ae7d P f7690ee176c94217927448be1f56d784: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 14:57:07.946344 17810 heartbeater.cc:503] Master 127.30.74.126:41513 was elected leader, sending a full tablet report...
I20260810 14:57:07.999816 18139 raft_consensus.cc:482] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:57:07.999990 18139 raft_consensus.cc:504] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7690ee176c94217927448be1f56d784" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38755 } } peers { permanent_uuid: "c2d2622adc8b4c89b890521abe858a88" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 45795 } } peers { permanent_uuid: "3369de05d48c460a8f315bd3cbf0403b" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35097 } }
I20260810 14:57:08.000512 18139 leader_election.cc:288] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7690ee176c94217927448be1f56d784 (127.30.74.65:38755), c2d2622adc8b4c89b890521abe858a88 (127.30.74.66:45795)
I20260810 14:57:08.003479 18128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:08.000849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:49881 (local address 127.30.74.66:45795)
0810 14:57:08.000899 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:08.000901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:08.000913 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:57:08.000948 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:08.000948 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:08.000960 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:08.001037 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:08.001209 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:08.001709 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:08.002711 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:08.003082 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:08.003171 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:08.003180 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:08.003360 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 14:57:08.003402 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 14:57:08.003479 18118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:08.000738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:45795 (local address 127.30.74.67:49881)
0810 14:57:08.000815 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:08.000829 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:08.000885 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:08.001043 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:08.001044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:08.001046 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:08.001125 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:08.001127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:08.001816 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:08.001817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:08.002600 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:08.002603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:08.003171 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:08.003172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:08.003267 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:08.003270 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:57:08.003350 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 14:57:08.003400 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 14:57:08.003942 17911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff845c95a88b4a5ea78b5ff42347ae7d" candidate_uuid: "3369de05d48c460a8f315bd3cbf0403b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2d2622adc8b4c89b890521abe858a88" is_pre_election: true
I20260810 14:57:08.003986 18129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:08.000817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:45429 (local address 127.30.74.65:38755)
0810 14:57:08.000888 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:08.000891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:08.001051 (+ 160us) server_negotiation.cc:407] Connection header received
0810 14:57:08.001167 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:08.001168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:08.001183 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:08.001258 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:08.001463 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:08.002007 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:08.003059 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:08.003452 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:08.003544 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:08.003552 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:08.003869 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 14:57:08.003916 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 14:57:08.003983 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:08.000751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:38755 (local address 127.30.74.67:45429)
0810 14:57:08.000970 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:08.000980 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:08.001070 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:08.001265 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:08.001266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:08.001268 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:08.001376 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:08.001378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:08.002105 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:08.002106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:08.002946 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:08.002949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:08.003685 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:08.003686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:08.003784 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:08.003789 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:57:08.003847 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 14:57:08.003893 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":120,"threads_started":1}
I20260810 14:57:08.004150 17911 raft_consensus.cc:2454] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3369de05d48c460a8f315bd3cbf0403b in term 0.
I20260810 14:57:08.004446 17755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff845c95a88b4a5ea78b5ff42347ae7d" candidate_uuid: "3369de05d48c460a8f315bd3cbf0403b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7690ee176c94217927448be1f56d784" is_pre_election: true
I20260810 14:57:08.004524 17983 leader_election.cc:302] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3369de05d48c460a8f315bd3cbf0403b, c2d2622adc8b4c89b890521abe858a88; no voters:
I20260810 14:57:08.004617 17755 raft_consensus.cc:2454] T ff845c95a88b4a5ea78b5ff42347ae7d P f7690ee176c94217927448be1f56d784 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3369de05d48c460a8f315bd3cbf0403b in term 0.
I20260810 14:57:08.004698 18139 raft_consensus.cc:2781] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:57:08.004805 18139 raft_consensus.cc:482] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:57:08.004844 18139 raft_consensus.cc:3037] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:08.005607 18139 raft_consensus.cc:504] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7690ee176c94217927448be1f56d784" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38755 } } peers { permanent_uuid: "c2d2622adc8b4c89b890521abe858a88" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 45795 } } peers { permanent_uuid: "3369de05d48c460a8f315bd3cbf0403b" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35097 } }
I20260810 14:57:08.005764 18139 leader_election.cc:288] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [CANDIDATE]: Term 1 election: Requested vote from peers f7690ee176c94217927448be1f56d784 (127.30.74.65:38755), c2d2622adc8b4c89b890521abe858a88 (127.30.74.66:45795)
I20260810 14:57:08.005977 17911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff845c95a88b4a5ea78b5ff42347ae7d" candidate_uuid: "3369de05d48c460a8f315bd3cbf0403b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2d2622adc8b4c89b890521abe858a88"
I20260810 14:57:08.005977 17755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff845c95a88b4a5ea78b5ff42347ae7d" candidate_uuid: "3369de05d48c460a8f315bd3cbf0403b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7690ee176c94217927448be1f56d784"
I20260810 14:57:08.006075 17911 raft_consensus.cc:3037] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:08.006081 17755 raft_consensus.cc:3037] T ff845c95a88b4a5ea78b5ff42347ae7d P f7690ee176c94217927448be1f56d784 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:08.007367 17911 raft_consensus.cc:2454] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3369de05d48c460a8f315bd3cbf0403b in term 1.
I20260810 14:57:08.007364 17755 raft_consensus.cc:2454] T ff845c95a88b4a5ea78b5ff42347ae7d P f7690ee176c94217927448be1f56d784 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3369de05d48c460a8f315bd3cbf0403b in term 1.
I20260810 14:57:08.007582 17984 leader_election.cc:302] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3369de05d48c460a8f315bd3cbf0403b, f7690ee176c94217927448be1f56d784; no voters:
I20260810 14:57:08.007689 18139 raft_consensus.cc:2781] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:08.007869 18139 raft_consensus.cc:686] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [term 1 LEADER]: Becoming Leader. State: Replica: 3369de05d48c460a8f315bd3cbf0403b, State: Running, Role: LEADER
I20260810 14:57:08.007977 18139 consensus_queue.cc:237] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7690ee176c94217927448be1f56d784" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38755 } } peers { permanent_uuid: "c2d2622adc8b4c89b890521abe858a88" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 45795 } } peers { permanent_uuid: "3369de05d48c460a8f315bd3cbf0403b" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35097 } }
I20260810 14:57:08.008980 17590 catalog_manager.cc:5495] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3369de05d48c460a8f315bd3cbf0403b (127.30.74.67). New cstate: current_term: 1 leader_uuid: "3369de05d48c460a8f315bd3cbf0403b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7690ee176c94217927448be1f56d784" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2d2622adc8b4c89b890521abe858a88" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 45795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3369de05d48c460a8f315bd3cbf0403b" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 35097 } health_report { overall_health: HEALTHY } } }
W20260810 14:57:08.023445 17963 tablet.cc:2369] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:57:08.031884 18130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:08.029177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38564 (local address 127.30.74.67:35097)
0810 14:57:08.029247 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:08.029249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:08.029258 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:57:08.029311 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:08.029312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:08.029331 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:08.029419 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:08.029584 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:08.030105 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:08.030904 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:08.031120 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:08.031215 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:08.031396 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:08.031522 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:08.031626 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:08.031647 (+ 21us) server_negotiation.cc:299] Negotiation successful
0810 14:57:08.031669 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 14:57:08.034631 17755 raft_consensus.cc:1261] T ff845c95a88b4a5ea78b5ff42347ae7d P f7690ee176c94217927448be1f56d784 [term 1 FOLLOWER]: Refusing update from remote peer 3369de05d48c460a8f315bd3cbf0403b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:57:08.034631 17911 raft_consensus.cc:1261] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88 [term 1 FOLLOWER]: Refusing update from remote peer 3369de05d48c460a8f315bd3cbf0403b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:57:08.035077 18142 consensus_queue.cc:1035] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7690ee176c94217927448be1f56d784" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:57:08.035246 18139 consensus_queue.cc:1035] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2d2622adc8b4c89b890521abe858a88" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 45795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:57:08.039047 18148 mvcc.cc:205] Tried to move back new op lower bound from 7316987199626506240 to 7316987199521542144. Current Snapshot: MvccSnapshot[applied={T|T < 7316987199626506240}]
I20260810 14:57:08.039366 18147 mvcc.cc:205] Tried to move back new op lower bound from 7316987199626506240 to 7316987199521542144. Current Snapshot: MvccSnapshot[applied={T|T < 7316987199626506240}]
I20260810 14:57:08.042483 17590 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:57:08.042608 17590 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:57:08.043514 17590 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fcdb7e151684f0891f2f47460b6348c: Sending DeleteTablet for 3 replicas of tablet ff845c95a88b4a5ea78b5ff42347ae7d
I20260810 14:57:08.043934 17887 tablet_service.cc:1514] Processing DeleteTablet for tablet ff845c95a88b4a5ea78b5ff42347ae7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:08 UTC) from {username='slave'} at 127.0.0.1:57158
I20260810 14:57:08.043931 17732 tablet_service.cc:1514] Processing DeleteTablet for tablet ff845c95a88b4a5ea78b5ff42347ae7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:08 UTC) from {username='slave'} at 127.0.0.1:59284
I20260810 14:57:08.044206 18157 tablet_replica.cc:330] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88: stopping tablet replica
I20260810 14:57:08.044365 18157 raft_consensus.cc:2229] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:57:08.044503 18040 tablet_service.cc:1514] Processing DeleteTablet for tablet ff845c95a88b4a5ea78b5ff42347ae7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:08 UTC) from {username='slave'} at 127.0.0.1:38562
I20260810 14:57:08.044498 18157 raft_consensus.cc:2258] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:57:08.044764 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 17662
I20260810 14:57:08.049206 18158 tablet_replica.cc:330] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b: stopping tablet replica
I20260810 14:57:08.049366 18158 raft_consensus.cc:2229] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:57:08.049500 18157 ts_tablet_manager.cc:1905] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:57:08.049556 18158 raft_consensus.cc:2258] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:57:08.051625 18157 ts_tablet_manager.cc:1918] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:57:08.051728 18157 log.cc:1196] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/ff845c95a88b4a5ea78b5ff42347ae7d
I20260810 14:57:08.052019 18157 ts_tablet_manager.cc:1939] T ff845c95a88b4a5ea78b5ff42347ae7d P c2d2622adc8b4c89b890521abe858a88: Deleting consensus metadata
I20260810 14:57:08.052629 17575 catalog_manager.cc:4841] TS c2d2622adc8b4c89b890521abe858a88 (127.30.74.66:45795): tablet ff845c95a88b4a5ea78b5ff42347ae7d (table test-table [id=a5c1cb6f331642f197eea44941039cef]) successfully deleted
W20260810 14:57:08.054760 17576 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:38755 (error 108)
I20260810 14:57:08.055006 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 17813
I20260810 14:57:08.056854 18158 ts_tablet_manager.cc:1905] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:57:08.059024 18158 ts_tablet_manager.cc:1918] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:57:08.059126 18158 log.cc:1196] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/ff845c95a88b4a5ea78b5ff42347ae7d
I20260810 14:57:08.059448 18158 ts_tablet_manager.cc:1939] T ff845c95a88b4a5ea78b5ff42347ae7d P 3369de05d48c460a8f315bd3cbf0403b: Deleting consensus metadata
I20260810 14:57:08.064483 17577 catalog_manager.cc:4841] TS 3369de05d48c460a8f315bd3cbf0403b (127.30.74.67:35097): tablet ff845c95a88b4a5ea78b5ff42347ae7d (table test-table [id=a5c1cb6f331642f197eea44941039cef]) successfully deleted
I20260810 14:57:08.064975 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 17965
I20260810 14:57:08.065282 18127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:08.065072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:38755 (local address 127.0.0.1:59288)
0810 14:57:08.065154 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:08.065205 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:38755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 14:57:08.067108 17576 catalog_manager.cc:4584] TS f7690ee176c94217927448be1f56d784 (127.30.74.65:38755): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff845c95a88b4a5ea78b5ff42347ae7d: Network error: Client connection negotiation failed: client connection to 127.30.74.65:38755: connect: Connection refused (error 111)
I20260810 14:57:08.073767 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 17557
2026-08-10T14:57:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (760 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:08 dist-test-slave-0b58 krb5kdc[18162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:08 dist-test-slave-0b58 krb5kdc[18162](info): set up 2 sockets
Aug 10 14:57:08 dist-test-slave-0b58 krb5kdc[18162](info): commencing operation
krb5kdc: starting...
W20260810 14:57:09.213310 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:09 dist-test-slave-0b58 krb5kdc[18162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:57:09.365942 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:34491
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:33213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:34491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:09.455981 18177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:09.456238 18177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:09.456287 18177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:09.457489 18177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:09.457540 18177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:09.457556 18177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:09.457568 18177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:09.457580 18177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:09.459110 18177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:34491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:34491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:09.459354 18177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:09.459553 18177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:09.465214 18182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:09.465173 18188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:09.465231 18177 server_base.cc:1034] running on GCE node
W20260810 14:57:09.465173 18183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:09.465803 18177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:09.466179 18177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:09.467386 18177 hybrid_clock.cc:648] HybridClock initialized: now 1786373829467379 us; error 59 us; skew 500 ppm
Aug 10 14:57:09 dist-test-slave-0b58 krb5kdc[18162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373829, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:09.470485 18177 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:57:09.471000 18177 webserver.cc:460] Webserver started at http://127.30.74.126:33681/ using document root <none> and password file <none>
I20260810 14:57:09.471251 18177 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:09.471335 18177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:09.471500 18177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:09.472760 18177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "d763138fd5d742a9889cb610242b7339"
format_stamp: "Formatted at 2026-08-10 14:57:09 on dist-test-slave-0b58"
server_key: "48c1d52fe5c3b66abc951ccbf322ef46"
server_key_iv: "337714ab96887e4ecb9cb789f4b06fc8"
server_key_version: "encryptionkey@0"
I20260810 14:57:09.473167 18177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "d763138fd5d742a9889cb610242b7339"
format_stamp: "Formatted at 2026-08-10 14:57:09 on dist-test-slave-0b58"
server_key: "48c1d52fe5c3b66abc951ccbf322ef46"
server_key_iv: "337714ab96887e4ecb9cb789f4b06fc8"
server_key_version: "encryptionkey@0"
I20260810 14:57:09.474781 18177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:57:09.475728 18197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:09.475948 18177 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:09.476085 18177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "d763138fd5d742a9889cb610242b7339"
format_stamp: "Formatted at 2026-08-10 14:57:09 on dist-test-slave-0b58"
server_key: "48c1d52fe5c3b66abc951ccbf322ef46"
server_key_iv: "337714ab96887e4ecb9cb789f4b06fc8"
server_key_version: "encryptionkey@0"
I20260810 14:57:09.476177 18177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:09.502817 18177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:09.505880 18177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:09.506109 18177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:09.510684 18177 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:34491
I20260810 14:57:09.510723 18257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:34491 every 8 connection(s)
I20260810 14:57:09.511148 18177 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:09.512153 18258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:09.513005 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 18177
I20260810 14:57:09.513171 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:09.513433 31017 external_mini_cluster.cc:1428] Setting key 62ebff05cfe99c4096bf36e1d908c56c
I20260810 14:57:09.515916 18258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339: Bootstrap starting.
I20260810 14:57:09.516669 18258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:09.517019 18258 log.cc:824] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:09.517912 18258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339: No bootstrap required, opened a new log
I20260810 14:57:09.520716 18258 raft_consensus.cc:348] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d763138fd5d742a9889cb610242b7339" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34491 } }
I20260810 14:57:09.520885 18258 raft_consensus.cc:374] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:09.520908 18258 raft_consensus.cc:729] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d763138fd5d742a9889cb610242b7339, State: Initialized, Role: FOLLOWER
I20260810 14:57:09.521044 18258 consensus_queue.cc:260] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d763138fd5d742a9889cb610242b7339" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34491 } }
I20260810 14:57:09.521142 18258 raft_consensus.cc:388] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:09.521188 18258 raft_consensus.cc:482] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:09.521262 18258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [term 0 FOLLOWER]: Advancing to term 1
Aug 10 14:57:09 dist-test-slave-0b58 krb5kdc[18162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:09.522179 18258 raft_consensus.cc:504] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d763138fd5d742a9889cb610242b7339" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34491 } }
I20260810 14:57:09.522320 18258 leader_election.cc:302] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d763138fd5d742a9889cb610242b7339; no voters:
I20260810 14:57:09.522538 18258 leader_election.cc:288] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:09.522614 18263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:09.522858 18258 sys_catalog.cc:564] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:09.523218 18263 raft_consensus.cc:686] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [term 1 LEADER]: Becoming Leader. State: Replica: d763138fd5d742a9889cb610242b7339, State: Running, Role: LEADER
I20260810 14:57:09.524327 18263 consensus_queue.cc:237] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d763138fd5d742a9889cb610242b7339" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34491 } }
I20260810 14:57:09.525005 18267 sys_catalog.cc:455] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d763138fd5d742a9889cb610242b7339. Latest consensus state: current_term: 1 leader_uuid: "d763138fd5d742a9889cb610242b7339" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d763138fd5d742a9889cb610242b7339" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34491 } } }
I20260810 14:57:09.525126 18267 sys_catalog.cc:458] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [sys.catalog]: This master's current role is: LEADER
W20260810 14:57:09.525821 18279 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:57:09.525879 18279 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:57:09.525928 18280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:09.526079 18280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:09.525005 18265 sys_catalog.cc:455] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d763138fd5d742a9889cb610242b7339" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d763138fd5d742a9889cb610242b7339" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34491 } } }
I20260810 14:57:09.526302 18265 sys_catalog.cc:458] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:09.526350 18261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:09.514055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60606 (local address 127.30.74.126:34491)
0810 14:57:09.514242 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:09.514250 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:09.514277 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:57:09.514746 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:09.514753 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:09.514808 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:09.514935 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:09.515879 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:09.517020 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:09.518105 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:09.518568 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:09.518663 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:09.522054 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:09.522059 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:09.522060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:09.522072 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:09.524140 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:09.524632 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:09.524636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:09.524637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:09.524759 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:09.525006 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:09.525007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:09.525008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:09.525366 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:09.525526 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:09.525901 (+ 375us) server_negotiation.cc:299] Negotiation successful
0810 14:57:09.526063 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260810 14:57:09.527784 18280 catalog_manager.cc:1347] Generated new cluster ID: cd9ab6acdaf345fa80ee07b7103d7ac0
I20260810 14:57:09.527846 18280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:09.538854 18280 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:09.539518 18280 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:09.543586 18280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339: Generated new TSK 0
I20260810 14:57:09.543805 18280 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:57:09.588913 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:34491
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:33213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:09.699865 18288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:09.700109 18288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:09.700136 18288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:09.701459 18288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:09.701519 18288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:09.701548 18288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:57:09.703230 18288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:34491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:09.703540 18288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:09.703812 18288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:09.709473 18294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:09.709477 18297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:09.709477 18295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:09.709511 18288 server_base.cc:1034] running on GCE node
I20260810 14:57:09.709908 18288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:09.710201 18288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:09.711392 18288 hybrid_clock.cc:648] HybridClock initialized: now 1786373829711364 us; error 41 us; skew 500 ppm
Aug 10 14:57:09 dist-test-slave-0b58 krb5kdc[18162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373829, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:09.714365 18288 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:57:09.714886 18288 webserver.cc:460] Webserver started at http://127.30.74.65:34111/ using document root <none> and password file <none>
I20260810 14:57:09.715145 18288 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:09.715217 18288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:09.715377 18288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:09.716627 18288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "1130d99f2c0f40ea964e218935c58b07"
format_stamp: "Formatted at 2026-08-10 14:57:09 on dist-test-slave-0b58"
server_key: "0ea6435a19b1054b9994efa051ed1f22"
server_key_iv: "ccb7763a327825bc878a9734909de3b6"
server_key_version: "encryptionkey@0"
I20260810 14:57:09.717012 18288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "1130d99f2c0f40ea964e218935c58b07"
format_stamp: "Formatted at 2026-08-10 14:57:09 on dist-test-slave-0b58"
server_key: "0ea6435a19b1054b9994efa051ed1f22"
server_key_iv: "ccb7763a327825bc878a9734909de3b6"
server_key_version: "encryptionkey@0"
I20260810 14:57:09.718569 18288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:57:09.719586 18307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:09.719945 18288 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:09.720085 18288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "1130d99f2c0f40ea964e218935c58b07"
format_stamp: "Formatted at 2026-08-10 14:57:09 on dist-test-slave-0b58"
server_key: "0ea6435a19b1054b9994efa051ed1f22"
server_key_iv: "ccb7763a327825bc878a9734909de3b6"
server_key_version: "encryptionkey@0"
I20260810 14:57:09.720182 18288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:09.733989 18288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:09.736737 18288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:09.736925 18288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:09.737192 18288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:09.737532 18288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:09.737586 18288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:09.737629 18288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:09.737661 18288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:09.744328 18288 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:40441
I20260810 14:57:09.744407 18429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:40441 every 8 connection(s)
I20260810 14:57:09.744802 18288 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:57:09.746958 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 18288
I20260810 14:57:09.747254 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:57:09.747720 31017 external_mini_cluster.cc:1428] Setting key 248c6970339b2f61b3bec58a7bc73508
Aug 10 14:57:09 dist-test-slave-0b58 krb5kdc[18162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373829, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
I20260810 14:57:09.760336 18261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:09.745780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:53583 (local address 127.30.74.126:34491)
0810 14:57:09.745865 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:09.745868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:09.746226 (+ 358us) server_negotiation.cc:407] Connection header received
0810 14:57:09.746952 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:09.746954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:09.746967 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:09.747035 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:09.749084 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:09.749647 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:09.750946 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:09.751174 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:09.751257 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:09.756508 (+ 5251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:09.756511 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:09.756511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:09.756521 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:09.758424 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:09.759058 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:09.759060 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:09.759061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:09.759137 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:09.759458 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:09.759459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:09.759459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:09.759620 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:09.759708 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:09.760049 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 14:57:09.760192 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 14:57:09.760515 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:09.745780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:34491 (local address 127.30.74.65:53583)
0810 14:57:09.746046 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:09.746107 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:09.746792 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:09.747300 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:09.747306 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:09.748622 (+ 1316us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:09.748959 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:09.748968 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:09.749759 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:09.749760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:09.750798 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:09.750802 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:09.751283 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:09.751285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:09.751481 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:09.754193 (+ 2712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:09.754207 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:09.756351 (+ 2144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:09.758589 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:09.758604 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:09.758607 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:09.758955 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:09.759226 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:09.759227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:09.759227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:09.759369 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:09.759700 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:09.759710 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:09.759885 (+ 175us) client_negotiation.cc:769] Sending connection context
0810 14:57:09.760010 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 14:57:09.760214 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":106,"threads_started":1}
I20260810 14:57:09.761510 18432 heartbeater.cc:347] Connected to a master server at 127.30.74.126:34491
I20260810 14:57:09.761649 18432 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:09.761897 18432 heartbeater.cc:511] Master 127.30.74.126:34491 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
I20260810 14:57:09.762648 18215 ts_manager.cc:194] Registered new tserver with Master: 1130d99f2c0f40ea964e218935c58b07 (127.30.74.65:40441)
I20260810 14:57:09.763427 18215 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:53583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:57:09.783273 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:34491
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:33213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:09.872968 18437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:09.873310 18437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:09.873390 18437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:09.874675 18437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:09.874788 18437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:09.874837 18437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:57:09.876579 18437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:34491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:09.876909 18437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:09.877198 18437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:09.883090 18442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:09.883090 18445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:09.883108 18443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:09.883108 18437 server_base.cc:1034] running on GCE node
I20260810 14:57:09.883534 18437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:09.883844 18437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:09.884999 18437 hybrid_clock.cc:648] HybridClock initialized: now 1786373829884972 us; error 40 us; skew 500 ppm
Aug 10 14:57:09 dist-test-slave-0b58 krb5kdc[18162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373829, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:09.887699 18437 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:57:09.888219 18437 webserver.cc:460] Webserver started at http://127.30.74.66:42381/ using document root <none> and password file <none>
I20260810 14:57:09.888523 18437 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:09.888577 18437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:09.888706 18437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:09.889849 18437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "19cfecbf076a45faaf618f3d6d32093d"
format_stamp: "Formatted at 2026-08-10 14:57:09 on dist-test-slave-0b58"
server_key: "3465bd2c1f8263572b551a52681337cc"
server_key_iv: "f0d1c6c9271cd9470bdcee46acd9e551"
server_key_version: "encryptionkey@0"
I20260810 14:57:09.890237 18437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "19cfecbf076a45faaf618f3d6d32093d"
format_stamp: "Formatted at 2026-08-10 14:57:09 on dist-test-slave-0b58"
server_key: "3465bd2c1f8263572b551a52681337cc"
server_key_iv: "f0d1c6c9271cd9470bdcee46acd9e551"
server_key_version: "encryptionkey@0"
I20260810 14:57:09.891710 18437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:57:09.892700 18452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:09.893010 18437 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:57:09.893108 18437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "19cfecbf076a45faaf618f3d6d32093d"
format_stamp: "Formatted at 2026-08-10 14:57:09 on dist-test-slave-0b58"
server_key: "3465bd2c1f8263572b551a52681337cc"
server_key_iv: "f0d1c6c9271cd9470bdcee46acd9e551"
server_key_version: "encryptionkey@0"
I20260810 14:57:09.893224 18437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:09.908592 18437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:09.911407 18437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:09.911594 18437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:09.911888 18437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:09.912336 18437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:09.912393 18437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:09.912439 18437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:09.912480 18437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:09.921586 18437 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:38103
I20260810 14:57:09.921681 18583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:38103 every 8 connection(s)
I20260810 14:57:09.922107 18437 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:57:09.930512 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 18437
I20260810 14:57:09.930788 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
Aug 10 14:57:09 dist-test-slave-0b58 krb5kdc[18162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373829, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:09.931071 31017 external_mini_cluster.cc:1428] Setting key 1e4f970635a8497d017f307842391de6
I20260810 14:57:09.935961 18261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:09.923146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:42193 (local address 127.30.74.126:34491)
0810 14:57:09.923230 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:09.923233 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:09.923583 (+ 350us) server_negotiation.cc:407] Connection header received
0810 14:57:09.924558 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:09.924560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:09.924577 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:09.924641 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:09.926402 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:09.927148 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:09.928462 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:09.928680 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:09.928788 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:09.931879 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:09.931881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:09.931881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:09.931892 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:09.933930 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:09.934523 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:09.934524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:09.934525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:09.934604 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:09.934934 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:09.934936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:09.934937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:09.935137 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:09.935229 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:09.935670 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 14:57:09.935789 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 14:57:09.935994 18586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:09.923131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:34491 (local address 127.30.74.66:42193)
0810 14:57:09.923344 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:09.923448 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:09.924412 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:09.924739 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:09.924744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:09.925798 (+ 1054us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:09.926234 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:09.926246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:09.927280 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:09.927281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:09.928358 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:09.928362 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:09.928821 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:09.928822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:09.928946 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:09.929867 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:09.929877 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:09.931511 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:09.934073 (+ 2562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:09.934086 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:09.934089 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:09.934418 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:09.934700 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:09.934701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:09.934702 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:09.934854 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:09.935330 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:09.935337 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:09.935494 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 14:57:09.935611 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 14:57:09.935795 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":141,"threads_started":1}
I20260810 14:57:09.936937 18584 heartbeater.cc:347] Connected to a master server at 127.30.74.126:34491
I20260810 14:57:09.937062 18584 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:09.937297 18584 heartbeater.cc:511] Master 127.30.74.126:34491 requested a full tablet report, sending...
I20260810 14:57:09.937808 18215 ts_manager.cc:194] Registered new tserver with Master: 19cfecbf076a45faaf618f3d6d32093d (127.30.74.66:38103)
I20260810 14:57:09.938215 18215 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:42193
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:57:09.968094 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:34491
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:33213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:10.060743 18589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:10.061031 18589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:10.061061 18589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:10.062549 18589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:10.062620 18589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:10.062652 18589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:57:10.064602 18589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:34491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:10.064869 18589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:10.065092 18589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:10.071452 18597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:10.071452 18595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:10.071910 18594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:10.072000 18589 server_base.cc:1034] running on GCE node
I20260810 14:57:10.072337 18589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:10.072764 18589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:10.073959 18589 hybrid_clock.cc:648] HybridClock initialized: now 1786373830073924 us; error 54 us; skew 500 ppm
Aug 10 14:57:10 dist-test-slave-0b58 krb5kdc[18162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373830, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:10.077087 18589 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:57:10.077807 18589 webserver.cc:460] Webserver started at http://127.30.74.67:36221/ using document root <none> and password file <none>
I20260810 14:57:10.078116 18589 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:10.078202 18589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:10.078379 18589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:10.079757 18589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "d53db4c41ecf400c91ce6e3283350d49"
format_stamp: "Formatted at 2026-08-10 14:57:10 on dist-test-slave-0b58"
server_key: "836cd1bc56a39b889ca6a05c231dcceb"
server_key_iv: "17cc097dc6ce28e9fc3a2b01626a6844"
server_key_version: "encryptionkey@0"
I20260810 14:57:10.080312 18589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "d53db4c41ecf400c91ce6e3283350d49"
format_stamp: "Formatted at 2026-08-10 14:57:10 on dist-test-slave-0b58"
server_key: "836cd1bc56a39b889ca6a05c231dcceb"
server_key_iv: "17cc097dc6ce28e9fc3a2b01626a6844"
server_key_version: "encryptionkey@0"
I20260810 14:57:10.082082 18589 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260810 14:57:10.083091 18607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:10.083341 18589 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:10.083467 18589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "d53db4c41ecf400c91ce6e3283350d49"
format_stamp: "Formatted at 2026-08-10 14:57:10 on dist-test-slave-0b58"
server_key: "836cd1bc56a39b889ca6a05c231dcceb"
server_key_iv: "17cc097dc6ce28e9fc3a2b01626a6844"
server_key_version: "encryptionkey@0"
I20260810 14:57:10.083556 18589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:10.099006 18589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:10.101676 18589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:10.101876 18589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:10.102187 18589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:10.102545 18589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:10.102602 18589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:10.102638 18589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:10.102669 18589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:10.109639 18589 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:33255
I20260810 14:57:10.109695 18741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:33255 every 8 connection(s)
I20260810 14:57:10.110061 18589 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:57:10.115562 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 18589
I20260810 14:57:10.115738 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:57:10.115994 31017 external_mini_cluster.cc:1428] Setting key a946fb967c89b1a2b68c8a760937e6c1
Aug 10 14:57:10 dist-test-slave-0b58 krb5kdc[18162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373830, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:10.121871 18261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.111074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:53921 (local address 127.30.74.126:34491)
0810 14:57:10.111160 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:10.111162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:10.111300 (+ 138us) server_negotiation.cc:407] Connection header received
0810 14:57:10.111883 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:10.111884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:10.111896 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:10.111956 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:10.113148 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.113668 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.114986 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.115282 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.115406 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.118447 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:10.118449 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:10.118450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:10.118461 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:10.120089 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:10.120678 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:10.120680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:10.120680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:10.120752 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:10.121085 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:10.121086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:10.121086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:10.121229 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:10.121324 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:10.121662 (+ 338us) server_negotiation.cc:299] Negotiation successful
0810 14:57:10.121761 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 14:57:10.122167 18744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.110881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:34491 (local address 127.30.74.67:53921)
0810 14:57:10.111144 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:10.111193 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:10.111780 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:10.112015 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:10.112019 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:10.112725 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:10.112948 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.112954 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.113848 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.113850 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.114815 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.114818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.115507 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.115508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.115655 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.116587 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:10.116597 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:10.118329 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:10.120212 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:10.120225 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:10.120228 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:10.120578 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:10.120855 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:10.120856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:10.120856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:10.120997 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:10.121314 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:10.121323 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:10.121501 (+ 178us) client_negotiation.cc:769] Sending connection context
0810 14:57:10.121623 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 14:57:10.121843 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":169,"threads_started":1}
I20260810 14:57:10.122921 18742 heartbeater.cc:347] Connected to a master server at 127.30.74.126:34491
I20260810 14:57:10.123024 18742 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:10.123313 18742 heartbeater.cc:511] Master 127.30.74.126:34491 requested a full tablet report, sending...
I20260810 14:57:10.123843 18215 ts_manager.cc:194] Registered new tserver with Master: d53db4c41ecf400c91ce6e3283350d49 (127.30.74.67:33255)
I20260810 14:57:10.124475 18215 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:53921
I20260810 14:57:10.128508 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:57:10.137545 18261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.129995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60610 (local address 127.30.74.126:34491)
0810 14:57:10.130061 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:10.130063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:10.130204 (+ 141us) server_negotiation.cc:407] Connection header received
0810 14:57:10.130299 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:10.130300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:10.130311 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:10.130364 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:10.130994 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.131515 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.132293 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.132503 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.132587 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.134303 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:10.134306 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:10.134306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:10.134317 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:10.135997 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:10.136599 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:10.136601 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:10.136601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:10.136670 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:10.136928 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:10.136929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:10.136930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:10.137098 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:10.137225 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:10.137350 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 14:57:10.137425 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 14:57:10.139828 18215 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:57:10.141776 18215 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:57:10.149742 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.144999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:38103 (local address 127.0.0.1:41334)
0810 14:57:10.145433 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:10.145444 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:10.145489 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:10.145958 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:10.145959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:10.145962 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:10.146007 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.146008 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.147312 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.147313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.148456 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.148459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.149416 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.149417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.149490 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.149504 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 14:57:10.149577 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 14:57:10.149627 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":70,"threads_started":1}
I20260810 14:57:10.149796 18758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.145239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41334 (local address 127.30.74.66:38103)
0810 14:57:10.145705 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:10.145710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:10.145721 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:57:10.145763 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:10.145765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:10.145780 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:10.145862 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:10.146138 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.147207 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.148554 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.149302 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.149386 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.149451 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:10.149611 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 14:57:10.149704 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":96,"threads_started":1}
I20260810 14:57:10.150420 18755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.144762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:33255 (local address 127.0.0.1:32806)
0810 14:57:10.144957 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:10.145001 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:10.145073 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:10.145521 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:10.145524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:10.145527 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:10.145615 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.145617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.147134 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.147135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.148736 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.148739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.149767 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.149768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.150211 (+ 443us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.150217 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:57:10.150279 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 14:57:10.150333 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":51,"thread_start_us":104,"threads_started":1}
I20260810 14:57:10.150892 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.145086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32806 (local address 127.30.74.67:33255)
0810 14:57:10.145312 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:10.145318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:10.145329 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:57:10.145375 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:10.145377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:10.145396 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:10.145719 (+ 323us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:10.145803 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.146995 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.148850 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.149649 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.149727 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.149785 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:10.150746 (+ 961us) server_negotiation.cc:299] Negotiation successful
0810 14:57:10.150818 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":104,"threads_started":1}
I20260810 14:57:10.150831 18510 tablet_service.cc:1467] Processing CreateTablet for tablet 9f0c61fc734c455cbdfa83b77b490630 (DEFAULT_TABLE table=test-table [id=6d734a4ee7bc4ec6bea7c073af6e94a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:10.151275 18510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f0c61fc734c455cbdfa83b77b490630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:10.151717 18666 tablet_service.cc:1467] Processing CreateTablet for tablet 9f0c61fc734c455cbdfa83b77b490630 (DEFAULT_TABLE table=test-table [id=6d734a4ee7bc4ec6bea7c073af6e94a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:10.152068 18666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f0c61fc734c455cbdfa83b77b490630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:10.154452 18761 tablet_bootstrap.cc:492] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d: Bootstrap starting.
I20260810 14:57:10.155323 18761 tablet_bootstrap.cc:654] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:10.155401 18760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.145314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33190 (local address 127.30.74.65:40441)
0810 14:57:10.145545 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:10.145550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:10.145564 (+ 14us) server_negotiation.cc:407] Connection header received
0810 14:57:10.145608 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:10.145610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:10.145628 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:10.145966 (+ 338us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:10.146052 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.146954 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.149835 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.154891 (+ 5056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.154994 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.155063 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:10.155236 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 14:57:10.155298 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":103,"threads_started":1}
I20260810 14:57:10.155409 18754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.144762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:40441 (local address 127.0.0.1:33190)
0810 14:57:10.144951 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:10.145003 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:10.145080 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:10.145768 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:10.145769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:10.145771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:10.145876 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.145879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.147088 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.147089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.148736 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.148739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.155023 (+ 6284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.155025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.155122 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.155128 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:57:10.155203 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 14:57:10.155266 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":60,"thread_start_us":79,"threads_started":1}
I20260810 14:57:10.155701 18762 tablet_bootstrap.cc:492] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49: Bootstrap starting.
I20260810 14:57:10.156361 18761 log.cc:824] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:10.156389 18762 tablet_bootstrap.cc:654] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:10.156219 18359 tablet_service.cc:1467] Processing CreateTablet for tablet 9f0c61fc734c455cbdfa83b77b490630 (DEFAULT_TABLE table=test-table [id=6d734a4ee7bc4ec6bea7c073af6e94a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:10.156652 18359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f0c61fc734c455cbdfa83b77b490630. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:10.157044 18762 log.cc:824] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:10.158124 18762 tablet_bootstrap.cc:492] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49: No bootstrap required, opened a new log
I20260810 14:57:10.158277 18762 ts_tablet_manager.cc:1397] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 14:57:10.159817 18766 tablet_bootstrap.cc:492] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07: Bootstrap starting.
I20260810 14:57:10.159751 18762 raft_consensus.cc:348] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53db4c41ecf400c91ce6e3283350d49" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 33255 } } peers { permanent_uuid: "1130d99f2c0f40ea964e218935c58b07" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40441 } } peers { permanent_uuid: "19cfecbf076a45faaf618f3d6d32093d" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38103 } }
I20260810 14:57:10.159945 18762 raft_consensus.cc:374] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:10.159996 18762 raft_consensus.cc:729] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d53db4c41ecf400c91ce6e3283350d49, State: Initialized, Role: FOLLOWER
I20260810 14:57:10.160162 18762 consensus_queue.cc:260] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53db4c41ecf400c91ce6e3283350d49" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 33255 } } peers { permanent_uuid: "1130d99f2c0f40ea964e218935c58b07" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40441 } } peers { permanent_uuid: "19cfecbf076a45faaf618f3d6d32093d" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38103 } }
I20260810 14:57:10.160549 18742 heartbeater.cc:503] Master 127.30.74.126:34491 was elected leader, sending a full tablet report...
I20260810 14:57:10.160615 18766 tablet_bootstrap.cc:654] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:10.160820 18761 tablet_bootstrap.cc:492] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d: No bootstrap required, opened a new log
I20260810 14:57:10.160549 18762 ts_tablet_manager.cc:1428] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260810 14:57:10.160913 18761 ts_tablet_manager.cc:1397] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260810 14:57:10.161376 18766 log.cc:824] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:10.162318 18766 tablet_bootstrap.cc:492] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07: No bootstrap required, opened a new log
I20260810 14:57:10.162410 18766 ts_tablet_manager.cc:1397] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 14:57:10.163229 18761 raft_consensus.cc:348] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53db4c41ecf400c91ce6e3283350d49" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 33255 } } peers { permanent_uuid: "1130d99f2c0f40ea964e218935c58b07" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40441 } } peers { permanent_uuid: "19cfecbf076a45faaf618f3d6d32093d" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38103 } }
I20260810 14:57:10.163442 18761 raft_consensus.cc:374] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:10.163481 18761 raft_consensus.cc:729] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cfecbf076a45faaf618f3d6d32093d, State: Initialized, Role: FOLLOWER
I20260810 14:57:10.163600 18761 consensus_queue.cc:260] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53db4c41ecf400c91ce6e3283350d49" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 33255 } } peers { permanent_uuid: "1130d99f2c0f40ea964e218935c58b07" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40441 } } peers { permanent_uuid: "19cfecbf076a45faaf618f3d6d32093d" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38103 } }
I20260810 14:57:10.163934 18761 ts_tablet_manager.cc:1428] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 14:57:10.163946 18584 heartbeater.cc:503] Master 127.30.74.126:34491 was elected leader, sending a full tablet report...
I20260810 14:57:10.164678 18766 raft_consensus.cc:348] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53db4c41ecf400c91ce6e3283350d49" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 33255 } } peers { permanent_uuid: "1130d99f2c0f40ea964e218935c58b07" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40441 } } peers { permanent_uuid: "19cfecbf076a45faaf618f3d6d32093d" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38103 } }
I20260810 14:57:10.164871 18766 raft_consensus.cc:374] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:10.164932 18766 raft_consensus.cc:729] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1130d99f2c0f40ea964e218935c58b07, State: Initialized, Role: FOLLOWER
I20260810 14:57:10.165066 18766 consensus_queue.cc:260] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53db4c41ecf400c91ce6e3283350d49" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 33255 } } peers { permanent_uuid: "1130d99f2c0f40ea964e218935c58b07" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40441 } } peers { permanent_uuid: "19cfecbf076a45faaf618f3d6d32093d" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38103 } }
I20260810 14:57:10.165416 18766 ts_tablet_manager.cc:1428] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260810 14:57:10.165423 18432 heartbeater.cc:503] Master 127.30.74.126:34491 was elected leader, sending a full tablet report...
W20260810 14:57:10.172861 18585 tablet.cc:2369] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:57:10.245682 18433 tablet.cc:2369] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:57:10.316972 18771 raft_consensus.cc:482] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:57:10.317175 18771 raft_consensus.cc:504] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53db4c41ecf400c91ce6e3283350d49" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 33255 } } peers { permanent_uuid: "1130d99f2c0f40ea964e218935c58b07" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40441 } } peers { permanent_uuid: "19cfecbf076a45faaf618f3d6d32093d" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38103 } }
I20260810 14:57:10.317585 18771 leader_election.cc:288] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d53db4c41ecf400c91ce6e3283350d49 (127.30.74.67:33255), 19cfecbf076a45faaf618f3d6d32093d (127.30.74.66:38103)
I20260810 14:57:10.321137 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.318033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:58117 (local address 127.30.74.67:33255)
0810 14:57:10.318102 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:10.318105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:10.318119 (+ 14us) server_negotiation.cc:407] Connection header received
0810 14:57:10.318157 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:10.318158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:10.318170 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:10.318258 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:10.318479 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.319029 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.320282 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.320735 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.320798 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.320807 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:10.321008 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 14:57:10.321052 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 14:57:10.321154 18772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.317864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:33255 (local address 127.30.74.65:58117)
0810 14:57:10.318001 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:10.318014 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:10.318075 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:10.318275 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:10.318276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:10.318279 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:10.318374 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.318376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.319128 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.319129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.320137 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.320141 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.320826 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.320827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.320915 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.320922 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 14:57:10.320986 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 14:57:10.321048 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260810 14:57:10.321293 18773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.318089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:38103 (local address 127.30.74.65:38817)
0810 14:57:10.318205 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:10.318216 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:10.318271 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:10.318478 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:10.318479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:10.318481 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:10.318556 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.318558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.319427 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.319428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.320259 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.320262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.321021 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.321022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.321092 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.321097 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:57:10.321155 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 14:57:10.321200 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
I20260810 14:57:10.321324 18758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.318291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:38817 (local address 127.30.74.66:38103)
0810 14:57:10.318346 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:10.318348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:10.318357 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:57:10.318385 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:10.318386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:10.318396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:10.318455 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:10.318689 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.319326 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.320386 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.320889 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.320981 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.320992 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:10.321189 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 14:57:10.321241 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 14:57:10.321604 18690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f0c61fc734c455cbdfa83b77b490630" candidate_uuid: "1130d99f2c0f40ea964e218935c58b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53db4c41ecf400c91ce6e3283350d49" is_pre_election: true
I20260810 14:57:10.321619 18531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f0c61fc734c455cbdfa83b77b490630" candidate_uuid: "1130d99f2c0f40ea964e218935c58b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cfecbf076a45faaf618f3d6d32093d" is_pre_election: true
I20260810 14:57:10.321800 18690 raft_consensus.cc:2454] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1130d99f2c0f40ea964e218935c58b07 in term 0.
I20260810 14:57:10.321815 18531 raft_consensus.cc:2454] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1130d99f2c0f40ea964e218935c58b07 in term 0.
I20260810 14:57:10.322094 18311 leader_election.cc:302] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1130d99f2c0f40ea964e218935c58b07, 19cfecbf076a45faaf618f3d6d32093d; no voters:
I20260810 14:57:10.322301 18771 raft_consensus.cc:2781] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:57:10.322393 18771 raft_consensus.cc:482] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:57:10.322429 18771 raft_consensus.cc:3037] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:10.323323 18771 raft_consensus.cc:504] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53db4c41ecf400c91ce6e3283350d49" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 33255 } } peers { permanent_uuid: "1130d99f2c0f40ea964e218935c58b07" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40441 } } peers { permanent_uuid: "19cfecbf076a45faaf618f3d6d32093d" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38103 } }
I20260810 14:57:10.323488 18771 leader_election.cc:288] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [CANDIDATE]: Term 1 election: Requested vote from peers d53db4c41ecf400c91ce6e3283350d49 (127.30.74.67:33255), 19cfecbf076a45faaf618f3d6d32093d (127.30.74.66:38103)
I20260810 14:57:10.323707 18690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f0c61fc734c455cbdfa83b77b490630" candidate_uuid: "1130d99f2c0f40ea964e218935c58b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d53db4c41ecf400c91ce6e3283350d49"
I20260810 14:57:10.323710 18531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f0c61fc734c455cbdfa83b77b490630" candidate_uuid: "1130d99f2c0f40ea964e218935c58b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cfecbf076a45faaf618f3d6d32093d"
I20260810 14:57:10.323817 18690 raft_consensus.cc:3037] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:10.323822 18531 raft_consensus.cc:3037] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:10.324750 18690 raft_consensus.cc:2454] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1130d99f2c0f40ea964e218935c58b07 in term 1.
I20260810 14:57:10.324927 18531 raft_consensus.cc:2454] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1130d99f2c0f40ea964e218935c58b07 in term 1.
I20260810 14:57:10.324962 18311 leader_election.cc:302] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1130d99f2c0f40ea964e218935c58b07, d53db4c41ecf400c91ce6e3283350d49; no voters:
I20260810 14:57:10.325075 18771 raft_consensus.cc:2781] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:10.325237 18771 raft_consensus.cc:686] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [term 1 LEADER]: Becoming Leader. State: Replica: 1130d99f2c0f40ea964e218935c58b07, State: Running, Role: LEADER
I20260810 14:57:10.325353 18771 consensus_queue.cc:237] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53db4c41ecf400c91ce6e3283350d49" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 33255 } } peers { permanent_uuid: "1130d99f2c0f40ea964e218935c58b07" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40441 } } peers { permanent_uuid: "19cfecbf076a45faaf618f3d6d32093d" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38103 } }
I20260810 14:57:10.326332 18214 catalog_manager.cc:5495] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1130d99f2c0f40ea964e218935c58b07 (127.30.74.65). New cstate: current_term: 1 leader_uuid: "1130d99f2c0f40ea964e218935c58b07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d53db4c41ecf400c91ce6e3283350d49" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 33255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1130d99f2c0f40ea964e218935c58b07" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 40441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19cfecbf076a45faaf618f3d6d32093d" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38103 } health_report { overall_health: UNKNOWN } } }
I20260810 14:57:10.334133 18760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.331031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33198 (local address 127.30.74.65:40441)
0810 14:57:10.331080 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:10.331082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:10.331090 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:57:10.331185 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:10.331186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:10.331203 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:10.331259 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:10.331539 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.332141 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.333361 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.333566 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.333661 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.333830 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:10.333926 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:10.334002 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:10.334054 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 14:57:10.334066 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:57:10.336741 18531 raft_consensus.cc:1261] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d [term 1 FOLLOWER]: Refusing update from remote peer 1130d99f2c0f40ea964e218935c58b07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:57:10.336863 18690 raft_consensus.cc:1261] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49 [term 1 FOLLOWER]: Refusing update from remote peer 1130d99f2c0f40ea964e218935c58b07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:57:10.337201 18771 consensus_queue.cc:1035] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cfecbf076a45faaf618f3d6d32093d" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 38103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:57:10.337488 18776 consensus_queue.cc:1035] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d53db4c41ecf400c91ce6e3283350d49" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 33255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:57:10.342983 18214 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:57:10.343120 18214 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:57:10.344053 18214 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d763138fd5d742a9889cb610242b7339: Sending DeleteTablet for 3 replicas of tablet 9f0c61fc734c455cbdfa83b77b490630
I20260810 14:57:10.344587 18359 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f0c61fc734c455cbdfa83b77b490630 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:10 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:33190
I20260810 14:57:10.344621 18510 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f0c61fc734c455cbdfa83b77b490630 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:10 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:41334
I20260810 14:57:10.344622 18666 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f0c61fc734c455cbdfa83b77b490630 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:10 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:32806
I20260810 14:57:10.344928 18791 tablet_replica.cc:330] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49: stopping tablet replica
I20260810 14:57:10.345055 18791 raft_consensus.cc:2229] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:57:10.345182 18791 raft_consensus.cc:2258] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:57:10.345120 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 18288
I20260810 14:57:10.345184 18789 tablet_replica.cc:330] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07: stopping tablet replica
I20260810 14:57:10.345504 18789 raft_consensus.cc:2229] T 9f0c61fc734c455cbdfa83b77b490630 P 1130d99f2c0f40ea964e218935c58b07 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:57:10.345963 18790 tablet_replica.cc:330] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d: stopping tablet replica
I20260810 14:57:10.346568 18790 raft_consensus.cc:2229] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:57:10.346756 18790 raft_consensus.cc:2258] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:57:10.347074 18791 ts_tablet_manager.cc:1905] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:57:10.347393 18790 ts_tablet_manager.cc:1905] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:57:10.348889 18791 ts_tablet_manager.cc:1918] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:57:10.348984 18791 log.cc:1196] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/9f0c61fc734c455cbdfa83b77b490630
I20260810 14:57:10.349313 18791 ts_tablet_manager.cc:1939] T 9f0c61fc734c455cbdfa83b77b490630 P d53db4c41ecf400c91ce6e3283350d49: Deleting consensus metadata
I20260810 14:57:10.349773 18201 catalog_manager.cc:4841] TS d53db4c41ecf400c91ce6e3283350d49 (127.30.74.67:33255): tablet 9f0c61fc734c455cbdfa83b77b490630 (table test-table [id=6d734a4ee7bc4ec6bea7c073af6e94a1]) successfully deleted
I20260810 14:57:10.350044 18790 ts_tablet_manager.cc:1918] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:57:10.350127 18790 log.cc:1196] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/9f0c61fc734c455cbdfa83b77b490630
I20260810 14:57:10.350394 18790 ts_tablet_manager.cc:1939] T 9f0c61fc734c455cbdfa83b77b490630 P 19cfecbf076a45faaf618f3d6d32093d: Deleting consensus metadata
I20260810 14:57:10.351197 18201 catalog_manager.cc:4841] TS 19cfecbf076a45faaf618f3d6d32093d (127.30.74.66:38103): tablet 9f0c61fc734c455cbdfa83b77b490630 (table test-table [id=6d734a4ee7bc4ec6bea7c073af6e94a1]) successfully deleted
W20260810 14:57:10.354799 18198 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:40441 (error 108)
I20260810 14:57:10.355100 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 18437
I20260810 14:57:10.360152 18754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.359944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:40441 (local address 127.0.0.1:33202)
0810 14:57:10.360037 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:10.360107 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:40441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 14:57:10.361212 18198 catalog_manager.cc:4584] TS 1130d99f2c0f40ea964e218935c58b07 (127.30.74.65:40441): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9f0c61fc734c455cbdfa83b77b490630: Network error: Client connection negotiation failed: client connection to 127.30.74.65:40441: connect: Connection refused (error 111)
I20260810 14:57:10.365478 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 18589
I20260810 14:57:10.373679 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 18177
2026-08-10T14:57:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2301 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T14:57:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:10Z Disabled control of system clock
I20260810 14:57:10.406944 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:34033
--webserver_interface=127.30.74.126
--webserver_port=0
--builtin_ntp_servers=127.30.74.84:33313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:34033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:57:10.496054 18796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:10.496356 18796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:10.496382 18796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:10.497830 18796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:10.497903 18796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:10.497922 18796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:10.497941 18796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:10.497958 18796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:10.500476 18796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:34033
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:34033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:10.500804 18796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:10.501223 18796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:10.507359 18801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:10.507368 18804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:10.507417 18802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:10.507534 18796 server_base.cc:1034] running on GCE node
I20260810 14:57:10.507874 18796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:10.508244 18796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:10.509505 18796 hybrid_clock.cc:648] HybridClock initialized: now 1786373830509493 us; error 34 us; skew 500 ppm
I20260810 14:57:10.515803 18796 webserver.cc:460] Webserver started at http://127.30.74.126:46577/ using document root <none> and password file <none>
I20260810 14:57:10.516152 18796 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:10.516214 18796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:10.516449 18796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:10.517701 18796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "880c5f5848644d9e84ca502acf020545"
format_stamp: "Formatted at 2026-08-10 14:57:10 on dist-test-slave-0b58"
server_key: "8fdcc80106f0d3292410794cb645c0d9"
server_key_iv: "e9adae40f0c07242f88dbeeb4d00bce7"
server_key_version: "encryptionkey@0"
I20260810 14:57:10.518105 18796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "880c5f5848644d9e84ca502acf020545"
format_stamp: "Formatted at 2026-08-10 14:57:10 on dist-test-slave-0b58"
server_key: "8fdcc80106f0d3292410794cb645c0d9"
server_key_iv: "e9adae40f0c07242f88dbeeb4d00bce7"
server_key_version: "encryptionkey@0"
I20260810 14:57:10.519639 18796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 14:57:10.520618 18810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:10.520824 18796 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:10.520932 18796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "880c5f5848644d9e84ca502acf020545"
format_stamp: "Formatted at 2026-08-10 14:57:10 on dist-test-slave-0b58"
server_key: "8fdcc80106f0d3292410794cb645c0d9"
server_key_iv: "e9adae40f0c07242f88dbeeb4d00bce7"
server_key_version: "encryptionkey@0"
I20260810 14:57:10.520998 18796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:10.533416 18796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:10.533736 18796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:10.533845 18796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:10.538054 18796 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:34033
I20260810 14:57:10.538089 18872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:34033 every 8 connection(s)
I20260810 14:57:10.538479 18796 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:10.539623 18875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:10.542250 18875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545: Bootstrap starting.
I20260810 14:57:10.542689 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 18796
I20260810 14:57:10.542788 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:10.543018 31017 external_mini_cluster.cc:1428] Setting key a5f6e22b2cdaf9030e3a53669c6feaf3
I20260810 14:57:10.543013 18875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:10.543401 18875 log.cc:824] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:10.544382 18875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545: No bootstrap required, opened a new log
I20260810 14:57:10.545816 18875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880c5f5848644d9e84ca502acf020545" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34033 } }
I20260810 14:57:10.545960 18875 raft_consensus.cc:374] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:10.545984 18875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 880c5f5848644d9e84ca502acf020545, State: Initialized, Role: FOLLOWER
I20260810 14:57:10.546149 18875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880c5f5848644d9e84ca502acf020545" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34033 } }
I20260810 14:57:10.546233 18875 raft_consensus.cc:388] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:10.546257 18875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:10.546324 18875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:10.546906 18875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880c5f5848644d9e84ca502acf020545" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34033 } }
I20260810 14:57:10.547034 18875 leader_election.cc:302] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 880c5f5848644d9e84ca502acf020545; no voters:
I20260810 14:57:10.547262 18875 leader_election.cc:288] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:10.547299 18880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:10.547417 18880 raft_consensus.cc:686] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [term 1 LEADER]: Becoming Leader. State: Replica: 880c5f5848644d9e84ca502acf020545, State: Running, Role: LEADER
I20260810 14:57:10.547506 18880 consensus_queue.cc:237] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880c5f5848644d9e84ca502acf020545" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34033 } }
I20260810 14:57:10.547572 18875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:10.547868 18880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 880c5f5848644d9e84ca502acf020545. Latest consensus state: current_term: 1 leader_uuid: "880c5f5848644d9e84ca502acf020545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880c5f5848644d9e84ca502acf020545" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34033 } } }
I20260810 14:57:10.547952 18880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:10.547881 18882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "880c5f5848644d9e84ca502acf020545" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "880c5f5848644d9e84ca502acf020545" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34033 } } }
I20260810 14:57:10.548130 18882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:10.548179 18885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:10.548779 18885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:10.550225 18878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.543646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50800 (local address 127.30.74.126:34033)
0810 14:57:10.543884 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:10.543892 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:10.543920 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:57:10.544480 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:10.544490 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:10.544563 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:10.544720 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:10.544975 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.546570 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.547579 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.548202 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.548342 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.549444 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:10.549448 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:10.549448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:10.549658 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:10.549716 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:10.549731 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:10.549770 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:10.549912 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 14:57:10.550000 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
I20260810 14:57:10.550746 18885 catalog_manager.cc:1347] Generated new cluster ID: 915fe2edf4e9490aa4b85a4c20ba44ec
I20260810 14:57:10.550805 18885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:10.560245 18885 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:10.561146 18885 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:10.567258 18885 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545: Generated new TSK 0
I20260810 14:57:10.567492 18885 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:10.572846 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:34033
--builtin_ntp_servers=127.30.74.84:33313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:57:10.660682 18901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:10.660921 18901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:10.660948 18901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:10.662231 18901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:10.662287 18901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:10.662315 18901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:57:10.663791 18901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:34033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:10.664103 18901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:10.664523 18901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:10.670149 18909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:10.670222 18907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:10.670192 18906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:10.670490 18901 server_base.cc:1034] running on GCE node
I20260810 14:57:10.670733 18901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:10.671095 18901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:10.672314 18901 hybrid_clock.cc:648] HybridClock initialized: now 1786373830672294 us; error 36 us; skew 500 ppm
I20260810 14:57:10.673998 18901 webserver.cc:460] Webserver started at http://127.30.74.65:45971/ using document root <none> and password file <none>
I20260810 14:57:10.674392 18901 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:10.674464 18901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:10.674616 18901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:10.676084 18901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "1812278958cb4c638d77d41c97ed6a3f"
format_stamp: "Formatted at 2026-08-10 14:57:10 on dist-test-slave-0b58"
server_key: "b3f7327ac7900cb24cd1bbceda99b552"
server_key_iv: "ed46e870065ed66d5c11301b60d21c35"
server_key_version: "encryptionkey@0"
I20260810 14:57:10.676558 18901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "1812278958cb4c638d77d41c97ed6a3f"
format_stamp: "Formatted at 2026-08-10 14:57:10 on dist-test-slave-0b58"
server_key: "b3f7327ac7900cb24cd1bbceda99b552"
server_key_iv: "ed46e870065ed66d5c11301b60d21c35"
server_key_version: "encryptionkey@0"
I20260810 14:57:10.678263 18901 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 14:57:10.679230 18915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:10.679457 18901 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:10.679565 18901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "1812278958cb4c638d77d41c97ed6a3f"
format_stamp: "Formatted at 2026-08-10 14:57:10 on dist-test-slave-0b58"
server_key: "b3f7327ac7900cb24cd1bbceda99b552"
server_key_iv: "ed46e870065ed66d5c11301b60d21c35"
server_key_version: "encryptionkey@0"
I20260810 14:57:10.679665 18901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:10.697068 18901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:10.697530 18901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:10.697717 18901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:10.698005 18901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:10.698433 18901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:10.698498 18901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:10.698542 18901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:10.698575 18901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:10.705264 18901 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:33933
I20260810 14:57:10.705324 19049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:33933 every 8 connection(s)
I20260810 14:57:10.705735 18901 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:57:10.709631 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 18901
I20260810 14:57:10.709774 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:57:10.709990 31017 external_mini_cluster.cc:1428] Setting key 99dd1850edba269866fb91e4f0b39f78
I20260810 14:57:10.711236 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:34033
--builtin_ntp_servers=127.30.74.84:33313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:57:10.712222 18878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.706627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:53055 (local address 127.30.74.126:34033)
0810 14:57:10.706758 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:10.706761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:10.706949 (+ 188us) server_negotiation.cc:407] Connection header received
0810 14:57:10.707530 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:10.707531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:10.707544 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:10.707597 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:10.708383 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.708921 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.710351 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.710652 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.710754 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.711911 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:10.711913 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:10.711913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:10.711919 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:10.711936 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:10.711940 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:10.712013 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:10.712107 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 14:57:10.712141 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 14:57:10.712504 19052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.706536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:34033 (local address 127.30.74.65:53055)
0810 14:57:10.706822 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:10.706859 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:10.707408 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:10.707673 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:10.707678 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:10.707687 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:10.708230 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.708234 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.709028 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.709029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.710227 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.710230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.710778 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.710779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.710943 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.711729 (+ 786us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:57:10.711729 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:10.711767 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:57:10.711794 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:10.712015 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:10.712022 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:10.712026 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:57:10.712102 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 14:57:10.712221 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":103,"threads_started":1}
I20260810 14:57:10.712980 19050 heartbeater.cc:347] Connected to a master server at 127.30.74.126:34033
I20260810 14:57:10.713107 19050 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:10.713320 19050 heartbeater.cc:511] Master 127.30.74.126:34033 requested a full tablet report, sending...
I20260810 14:57:10.713896 18829 ts_manager.cc:194] Registered new tserver with Master: 1812278958cb4c638d77d41c97ed6a3f (127.30.74.65:33933)
I20260810 14:57:10.714555 18829 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.74.65:53055
W20260810 14:57:10.798712 19053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:10.798949 19053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:10.798972 19053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:10.800231 19053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:10.800320 19053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:10.800349 19053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:57:10.801852 19053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:34033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:10.802148 19053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:10.802503 19053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:10.807940 19059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:10.807934 19060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:10.807932 19063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:10.808109 19053 server_base.cc:1034] running on GCE node
I20260810 14:57:10.808387 19053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:10.808698 19053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:10.809857 19053 hybrid_clock.cc:648] HybridClock initialized: now 1786373830809848 us; error 30 us; skew 500 ppm
I20260810 14:57:10.811264 19053 webserver.cc:460] Webserver started at http://127.30.74.66:45479/ using document root <none> and password file <none>
I20260810 14:57:10.811565 19053 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:10.811646 19053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:10.811811 19053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:10.813047 19053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "5dc4f925fbca4d4d884ba8849b7bd39f"
format_stamp: "Formatted at 2026-08-10 14:57:10 on dist-test-slave-0b58"
server_key: "3ed33d6abd9bda557dbeb4a1dd2e21e5"
server_key_iv: "29e56259cf30cab9b75930914d9fef23"
server_key_version: "encryptionkey@0"
I20260810 14:57:10.813556 19053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "5dc4f925fbca4d4d884ba8849b7bd39f"
format_stamp: "Formatted at 2026-08-10 14:57:10 on dist-test-slave-0b58"
server_key: "3ed33d6abd9bda557dbeb4a1dd2e21e5"
server_key_iv: "29e56259cf30cab9b75930914d9fef23"
server_key_version: "encryptionkey@0"
I20260810 14:57:10.815055 19053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 14:57:10.815964 19069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:10.816224 19053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:10.816380 19053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "5dc4f925fbca4d4d884ba8849b7bd39f"
format_stamp: "Formatted at 2026-08-10 14:57:10 on dist-test-slave-0b58"
server_key: "3ed33d6abd9bda557dbeb4a1dd2e21e5"
server_key_iv: "29e56259cf30cab9b75930914d9fef23"
server_key_version: "encryptionkey@0"
I20260810 14:57:10.816478 19053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:10.825414 19053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:10.825773 19053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:10.825935 19053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:10.826207 19053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:10.826571 19053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:10.826628 19053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:10.826675 19053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:10.826696 19053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:10.833166 19053 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:42927
I20260810 14:57:10.833211 19206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:42927 every 8 connection(s)
I20260810 14:57:10.833596 19053 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:57:10.838740 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 19053
I20260810 14:57:10.838888 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:57:10.839120 31017 external_mini_cluster.cc:1428] Setting key 14f9174097b1f07f57949e8bf7040bcf
I20260810 14:57:10.839584 18878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.834405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:48285 (local address 127.30.74.126:34033)
0810 14:57:10.834468 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:10.834470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:10.834748 (+ 278us) server_negotiation.cc:407] Connection header received
0810 14:57:10.835313 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:10.835314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:10.835327 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:10.835397 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:10.836197 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.836802 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.837857 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.838064 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.838167 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.839249 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:10.839250 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:10.839250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:10.839257 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:10.839279 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:10.839284 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:10.839358 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:10.839468 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 14:57:10.839503 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 14:57:10.839849 19209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.834383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:34033 (local address 127.30.74.66:48285)
0810 14:57:10.834573 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:10.834618 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:10.835144 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:10.835484 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:10.835487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:10.835497 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:10.836036 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.836041 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.836902 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.836903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.837753 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.837755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.838205 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.838206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.838332 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.839059 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:57:10.839059 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:10.839092 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:57:10.839117 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:10.839365 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:10.839373 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:10.839377 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:57:10.839454 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 14:57:10.839598 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260810 14:57:10.840313 19207 heartbeater.cc:347] Connected to a master server at 127.30.74.126:34033
I20260810 14:57:10.840354 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:34033
--builtin_ntp_servers=127.30.74.84:33313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:57:10.840433 19207 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:10.840668 19207 heartbeater.cc:511] Master 127.30.74.126:34033 requested a full tablet report, sending...
I20260810 14:57:10.841082 18829 ts_manager.cc:194] Registered new tserver with Master: 5dc4f925fbca4d4d884ba8849b7bd39f (127.30.74.66:42927)
I20260810 14:57:10.841497 18829 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.74.66:48285
W20260810 14:57:10.930235 19210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:10.930482 19210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:10.930507 19210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:10.931708 19210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:10.931751 19210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:10.931777 19210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:57:10.933334 19210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:34033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:10.933579 19210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:10.933884 19210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:10.939529 19215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:10.939536 19216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:10.939555 19218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:10.939770 19210 server_base.cc:1034] running on GCE node
I20260810 14:57:10.940111 19210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:10.940441 19210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:10.941648 19210 hybrid_clock.cc:648] HybridClock initialized: now 1786373830941646 us; error 48 us; skew 500 ppm
I20260810 14:57:10.943148 19210 webserver.cc:460] Webserver started at http://127.30.74.67:46335/ using document root <none> and password file <none>
I20260810 14:57:10.943466 19210 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:10.943544 19210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:10.943717 19210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:10.944964 19210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "d37696ed894042c39ea41a0e161ea8cb"
format_stamp: "Formatted at 2026-08-10 14:57:10 on dist-test-slave-0b58"
server_key: "ca6c73c0a58c475801758d91bbf96146"
server_key_iv: "7e2806ae9aa320833f15858aa568ebdf"
server_key_version: "encryptionkey@0"
I20260810 14:57:10.945362 19210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "d37696ed894042c39ea41a0e161ea8cb"
format_stamp: "Formatted at 2026-08-10 14:57:10 on dist-test-slave-0b58"
server_key: "ca6c73c0a58c475801758d91bbf96146"
server_key_iv: "7e2806ae9aa320833f15858aa568ebdf"
server_key_version: "encryptionkey@0"
I20260810 14:57:10.946885 19210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:57:10.947726 19225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:10.947949 19210 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:10.948091 19210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "d37696ed894042c39ea41a0e161ea8cb"
format_stamp: "Formatted at 2026-08-10 14:57:10 on dist-test-slave-0b58"
server_key: "ca6c73c0a58c475801758d91bbf96146"
server_key_iv: "7e2806ae9aa320833f15858aa568ebdf"
server_key_version: "encryptionkey@0"
I20260810 14:57:10.948180 19210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:10.968079 19210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:10.968552 19210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:10.968729 19210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:10.968995 19210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:10.969380 19210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:10.969446 19210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:10.969491 19210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:10.969534 19210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:10.975732 19210 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:42151
I20260810 14:57:10.975782 19358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:42151 every 8 connection(s)
I20260810 14:57:10.976152 19210 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:57:10.977401 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 19210
I20260810 14:57:10.977519 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:57:10.977746 31017 external_mini_cluster.cc:1428] Setting key e04659ea8fa66d722b5fa7bb91d34b6c
I20260810 14:57:10.982342 18878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.976961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:55143 (local address 127.30.74.126:34033)
0810 14:57:10.977056 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:10.977059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:10.977369 (+ 310us) server_negotiation.cc:407] Connection header received
0810 14:57:10.978201 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:10.978202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:10.978216 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:10.978296 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:10.979228 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.979723 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.980849 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.981071 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.981152 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.982069 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:10.982070 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:10.982071 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:10.982077 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:10.982099 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:10.982109 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:10.982161 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:10.982251 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 14:57:10.982278 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 14:57:10.982570 19361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.976933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:34033 (local address 127.30.74.67:55143)
0810 14:57:10.977187 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:10.977240 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:10.978004 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:10.978415 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:10.978420 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:10.978431 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:10.979141 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.979147 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.979815 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.979815 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.980747 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:10.980750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.981160 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.981161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:10.981279 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.981865 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:57:10.981866 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:10.981922 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:57:10.981954 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:10.982171 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:10.982176 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:10.982179 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:57:10.982237 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 14:57:10.982343 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":119,"threads_started":1}
I20260810 14:57:10.983211 19359 heartbeater.cc:347] Connected to a master server at 127.30.74.126:34033
I20260810 14:57:10.983318 19359 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:10.983556 19359 heartbeater.cc:511] Master 127.30.74.126:34033 requested a full tablet report, sending...
I20260810 14:57:10.984006 18829 ts_manager.cc:194] Registered new tserver with Master: d37696ed894042c39ea41a0e161ea8cb (127.30.74.67:42151)
I20260810 14:57:10.984457 18829 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.74.67:55143
I20260810 14:57:10.989729 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:57:10.994452 18878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:10.991408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42554 (local address 127.30.74.126:34033)
0810 14:57:10.991475 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:10.991477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:10.991631 (+ 154us) server_negotiation.cc:407] Connection header received
0810 14:57:10.991680 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:10.991681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:10.991692 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:10.991758 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:10.991902 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.992450 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.993201 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:10.993410 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:10.993466 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:10.994174 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:10.994175 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:10.994175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:10.994180 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:10.994206 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:10.994210 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:10.994305 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:10.994375 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 14:57:10.994387 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 14:57:10.996503 18829 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:57:10.998333 18829 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:57:11.005430 19370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:11.000821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:42151 (local address 127.0.0.1:36102)
0810 14:57:11.001588 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:11.001704 (+ 116us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:11.001759 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:11.001953 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:11.001954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:11.001956 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:11.001987 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:11.001988 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.003300 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.003301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:11.004309 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:11.004311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.005096 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.005097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:11.005186 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:11.005201 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 14:57:11.005281 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 14:57:11.005342 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"mutex_wait_us":4,"spinlock_wait_cycles":768,"thread_start_us":116,"threads_started":1}
I20260810 14:57:11.005446 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:11.001118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36102 (local address 127.30.74.67:42151)
0810 14:57:11.001347 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:11.001351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:11.001800 (+ 449us) server_negotiation.cc:407] Connection header received
0810 14:57:11.001850 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:11.001853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:11.001869 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:11.001952 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:11.002046 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.003190 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.004399 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.004984 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.005092 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:11.005213 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:11.005299 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 14:57:11.005341 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":125,"threads_started":1}
I20260810 14:57:11.005580 19367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:11.000821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:33933 (local address 127.0.0.1:35746)
0810 14:57:11.001394 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:11.001413 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:11.001494 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:11.001748 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:11.001749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:11.001751 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:11.001806 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:11.001808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.002770 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.002771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:11.003938 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:11.003941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.005318 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.005319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:11.005384 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:11.005388 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:57:11.005440 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 14:57:11.005484 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":1,"thread_start_us":100,"threads_started":1}
I20260810 14:57:11.005764 19371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:11.000924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53906 (local address 127.30.74.66:42927)
0810 14:57:11.001090 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:11.001096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:11.001248 (+ 152us) server_negotiation.cc:407] Connection header received
0810 14:57:11.001367 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:11.001369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:11.001386 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:11.001480 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:11.001656 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.003202 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.004525 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.005264 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.005331 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:11.005409 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:11.005622 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 14:57:11.005666 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
I20260810 14:57:11.005884 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:11.001117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35746 (local address 127.30.74.65:33933)
0810 14:57:11.001336 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:11.001340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:11.001484 (+ 144us) server_negotiation.cc:407] Connection header received
0810 14:57:11.001571 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:11.001574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:11.001590 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:11.001669 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:11.001873 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.002676 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.004283 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.005088 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.005168 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:11.005277 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:11.005739 (+ 462us) server_negotiation.cc:299] Negotiation successful
0810 14:57:11.005791 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":91,"threads_started":1}
I20260810 14:57:11.005681 19368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:11.000820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:42927 (local address 127.0.0.1:53906)
0810 14:57:11.001118 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:11.001159 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:11.001262 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:11.001486 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:11.001489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:11.001491 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:11.001567 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:11.001569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.003293 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.003295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:11.004429 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:11.004432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.005442 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.005443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:11.005494 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:11.005496 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 14:57:11.005541 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 14:57:11.005573 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":22,"thread_start_us":121,"threads_started":1}
I20260810 14:57:11.006585 18973 tablet_service.cc:1467] Processing CreateTablet for tablet 044b9098f6bc4cfeb5936ec09ed1b771 (DEFAULT_TABLE table=test-table [id=d9e8b959a43a4101bd205838d0bd9b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:11.006603 19282 tablet_service.cc:1467] Processing CreateTablet for tablet 044b9098f6bc4cfeb5936ec09ed1b771 (DEFAULT_TABLE table=test-table [id=d9e8b959a43a4101bd205838d0bd9b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:11.006762 19126 tablet_service.cc:1467] Processing CreateTablet for tablet 044b9098f6bc4cfeb5936ec09ed1b771 (DEFAULT_TABLE table=test-table [id=d9e8b959a43a4101bd205838d0bd9b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:11.006916 18973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 044b9098f6bc4cfeb5936ec09ed1b771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:11.006975 19282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 044b9098f6bc4cfeb5936ec09ed1b771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:11.007056 19126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 044b9098f6bc4cfeb5936ec09ed1b771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:11.010314 19374 tablet_bootstrap.cc:492] T 044b9098f6bc4cfeb5936ec09ed1b771 P 1812278958cb4c638d77d41c97ed6a3f: Bootstrap starting.
I20260810 14:57:11.010851 19376 tablet_bootstrap.cc:492] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f: Bootstrap starting.
I20260810 14:57:11.011258 19374 tablet_bootstrap.cc:654] T 044b9098f6bc4cfeb5936ec09ed1b771 P 1812278958cb4c638d77d41c97ed6a3f: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:11.011595 19374 log.cc:824] T 044b9098f6bc4cfeb5936ec09ed1b771 P 1812278958cb4c638d77d41c97ed6a3f: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:11.011612 19376 tablet_bootstrap.cc:654] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:11.011970 19376 log.cc:824] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:11.012401 19375 tablet_bootstrap.cc:492] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb: Bootstrap starting.
I20260810 14:57:11.012550 19374 tablet_bootstrap.cc:492] T 044b9098f6bc4cfeb5936ec09ed1b771 P 1812278958cb4c638d77d41c97ed6a3f: No bootstrap required, opened a new log
I20260810 14:57:11.013011 19374 ts_tablet_manager.cc:1397] T 044b9098f6bc4cfeb5936ec09ed1b771 P 1812278958cb4c638d77d41c97ed6a3f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 14:57:11.013155 19375 tablet_bootstrap.cc:654] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:11.013404 19376 tablet_bootstrap.cc:492] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f: No bootstrap required, opened a new log
I20260810 14:57:11.013470 19376 ts_tablet_manager.cc:1397] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 14:57:11.013485 19375 log.cc:824] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:11.014312 19375 tablet_bootstrap.cc:492] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb: No bootstrap required, opened a new log
I20260810 14:57:11.014384 19375 ts_tablet_manager.cc:1397] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 14:57:11.015568 19374 raft_consensus.cc:348] T 044b9098f6bc4cfeb5936ec09ed1b771 P 1812278958cb4c638d77d41c97ed6a3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1812278958cb4c638d77d41c97ed6a3f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33933 } } peers { permanent_uuid: "5dc4f925fbca4d4d884ba8849b7bd39f" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 42927 } } peers { permanent_uuid: "d37696ed894042c39ea41a0e161ea8cb" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42151 } }
I20260810 14:57:11.015738 19374 raft_consensus.cc:374] T 044b9098f6bc4cfeb5936ec09ed1b771 P 1812278958cb4c638d77d41c97ed6a3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:11.015767 19374 raft_consensus.cc:729] T 044b9098f6bc4cfeb5936ec09ed1b771 P 1812278958cb4c638d77d41c97ed6a3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1812278958cb4c638d77d41c97ed6a3f, State: Initialized, Role: FOLLOWER
I20260810 14:57:11.015906 19374 consensus_queue.cc:260] T 044b9098f6bc4cfeb5936ec09ed1b771 P 1812278958cb4c638d77d41c97ed6a3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1812278958cb4c638d77d41c97ed6a3f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33933 } } peers { permanent_uuid: "5dc4f925fbca4d4d884ba8849b7bd39f" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 42927 } } peers { permanent_uuid: "d37696ed894042c39ea41a0e161ea8cb" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42151 } }
I20260810 14:57:11.016151 19374 ts_tablet_manager.cc:1428] T 044b9098f6bc4cfeb5936ec09ed1b771 P 1812278958cb4c638d77d41c97ed6a3f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:57:11.016319 19376 raft_consensus.cc:348] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1812278958cb4c638d77d41c97ed6a3f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33933 } } peers { permanent_uuid: "5dc4f925fbca4d4d884ba8849b7bd39f" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 42927 } } peers { permanent_uuid: "d37696ed894042c39ea41a0e161ea8cb" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42151 } }
I20260810 14:57:11.016494 19376 raft_consensus.cc:374] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:11.016528 19376 raft_consensus.cc:729] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc4f925fbca4d4d884ba8849b7bd39f, State: Initialized, Role: FOLLOWER
I20260810 14:57:11.016672 19376 consensus_queue.cc:260] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1812278958cb4c638d77d41c97ed6a3f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33933 } } peers { permanent_uuid: "5dc4f925fbca4d4d884ba8849b7bd39f" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 42927 } } peers { permanent_uuid: "d37696ed894042c39ea41a0e161ea8cb" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42151 } }
I20260810 14:57:11.016954 19376 ts_tablet_manager.cc:1428] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 14:57:11.017069 19207 heartbeater.cc:503] Master 127.30.74.126:34033 was elected leader, sending a full tablet report...
I20260810 14:57:11.017216 19375 raft_consensus.cc:348] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1812278958cb4c638d77d41c97ed6a3f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33933 } } peers { permanent_uuid: "5dc4f925fbca4d4d884ba8849b7bd39f" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 42927 } } peers { permanent_uuid: "d37696ed894042c39ea41a0e161ea8cb" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42151 } }
I20260810 14:57:11.017370 19375 raft_consensus.cc:374] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:11.017409 19375 raft_consensus.cc:729] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d37696ed894042c39ea41a0e161ea8cb, State: Initialized, Role: FOLLOWER
I20260810 14:57:11.017765 19375 consensus_queue.cc:260] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1812278958cb4c638d77d41c97ed6a3f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33933 } } peers { permanent_uuid: "5dc4f925fbca4d4d884ba8849b7bd39f" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 42927 } } peers { permanent_uuid: "d37696ed894042c39ea41a0e161ea8cb" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42151 } }
I20260810 14:57:11.018052 19375 ts_tablet_manager.cc:1428] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260810 14:57:11.018172 19359 heartbeater.cc:503] Master 127.30.74.126:34033 was elected leader, sending a full tablet report...
I20260810 14:57:11.020702 19050 heartbeater.cc:503] Master 127.30.74.126:34033 was elected leader, sending a full tablet report...
W20260810 14:57:11.084175 19208 tablet.cc:2369] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:57:11.189689 19381 raft_consensus.cc:482] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:57:11.189853 19381 raft_consensus.cc:504] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1812278958cb4c638d77d41c97ed6a3f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33933 } } peers { permanent_uuid: "5dc4f925fbca4d4d884ba8849b7bd39f" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 42927 } } peers { permanent_uuid: "d37696ed894042c39ea41a0e161ea8cb" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42151 } }
I20260810 14:57:11.190374 19381 leader_election.cc:288] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1812278958cb4c638d77d41c97ed6a3f (127.30.74.65:33933), d37696ed894042c39ea41a0e161ea8cb (127.30.74.67:42151)
I20260810 14:57:11.193632 19209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:11.190513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:33933 (local address 127.30.74.66:56133)
0810 14:57:11.190592 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:11.190607 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:11.190664 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:11.190833 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:11.190834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:11.190836 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:11.190913 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:11.190916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.191753 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.191754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:11.192716 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:11.192719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.193328 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.193330 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:11.193407 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:11.193412 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:57:11.193490 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 14:57:11.193544 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 14:57:11.193717 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:11.190602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:56133 (local address 127.30.74.65:33933)
0810 14:57:11.190672 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:11.190674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:11.190713 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:57:11.190744 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:11.190745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:11.190758 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:11.190820 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:11.191013 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.191675 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.192827 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.193237 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.193294 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:11.193304 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:11.193574 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 14:57:11.193624 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 14:57:11.193938 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:11.190745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:59721 (local address 127.30.74.67:42151)
0810 14:57:11.190806 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:11.190808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:11.190918 (+ 110us) server_negotiation.cc:407] Connection header received
0810 14:57:11.190972 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:11.190973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:11.190985 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:11.191046 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:11.191217 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.191865 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.192966 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.193489 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.193557 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:11.193565 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:11.193801 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 14:57:11.193857 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 14:57:11.194087 18997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044b9098f6bc4cfeb5936ec09ed1b771" candidate_uuid: "5dc4f925fbca4d4d884ba8849b7bd39f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1812278958cb4c638d77d41c97ed6a3f" is_pre_election: true
I20260810 14:57:11.194202 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:11.190653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:42151 (local address 127.30.74.66:59721)
0810 14:57:11.190844 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:11.190855 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:11.190913 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:11.191068 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:11.191069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:11.191070 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:11.191144 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:11.191146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.191957 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.191958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:11.192854 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:11.192857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.193595 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.193596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:11.193680 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:11.193684 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:57:11.194068 (+ 384us) client_negotiation.cc:240] Negotiation successful
0810 14:57:11.194118 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":107,"threads_started":1}
I20260810 14:57:11.194294 18997 raft_consensus.cc:2454] T 044b9098f6bc4cfeb5936ec09ed1b771 P 1812278958cb4c638d77d41c97ed6a3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dc4f925fbca4d4d884ba8849b7bd39f in term 0.
I20260810 14:57:11.194561 19071 leader_election.cc:302] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1812278958cb4c638d77d41c97ed6a3f, 5dc4f925fbca4d4d884ba8849b7bd39f; no voters:
I20260810 14:57:11.194582 19307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044b9098f6bc4cfeb5936ec09ed1b771" candidate_uuid: "5dc4f925fbca4d4d884ba8849b7bd39f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d37696ed894042c39ea41a0e161ea8cb" is_pre_election: true
I20260810 14:57:11.194721 19381 raft_consensus.cc:2781] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:57:11.194754 19307 raft_consensus.cc:2454] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5dc4f925fbca4d4d884ba8849b7bd39f in term 0.
I20260810 14:57:11.194787 19381 raft_consensus.cc:482] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:57:11.194823 19381 raft_consensus.cc:3037] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:11.195607 19381 raft_consensus.cc:504] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1812278958cb4c638d77d41c97ed6a3f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33933 } } peers { permanent_uuid: "5dc4f925fbca4d4d884ba8849b7bd39f" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 42927 } } peers { permanent_uuid: "d37696ed894042c39ea41a0e161ea8cb" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42151 } }
I20260810 14:57:11.195824 19381 leader_election.cc:288] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [CANDIDATE]: Term 1 election: Requested vote from peers 1812278958cb4c638d77d41c97ed6a3f (127.30.74.65:33933), d37696ed894042c39ea41a0e161ea8cb (127.30.74.67:42151)
I20260810 14:57:11.195945 18997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044b9098f6bc4cfeb5936ec09ed1b771" candidate_uuid: "5dc4f925fbca4d4d884ba8849b7bd39f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1812278958cb4c638d77d41c97ed6a3f"
I20260810 14:57:11.196005 19307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044b9098f6bc4cfeb5936ec09ed1b771" candidate_uuid: "5dc4f925fbca4d4d884ba8849b7bd39f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d37696ed894042c39ea41a0e161ea8cb"
I20260810 14:57:11.196030 18997 raft_consensus.cc:3037] T 044b9098f6bc4cfeb5936ec09ed1b771 P 1812278958cb4c638d77d41c97ed6a3f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:11.196095 19307 raft_consensus.cc:3037] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:11.196878 18997 raft_consensus.cc:2454] T 044b9098f6bc4cfeb5936ec09ed1b771 P 1812278958cb4c638d77d41c97ed6a3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dc4f925fbca4d4d884ba8849b7bd39f in term 1.
I20260810 14:57:11.197049 19071 leader_election.cc:302] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1812278958cb4c638d77d41c97ed6a3f, 5dc4f925fbca4d4d884ba8849b7bd39f; no voters:
I20260810 14:57:11.197063 19307 raft_consensus.cc:2454] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5dc4f925fbca4d4d884ba8849b7bd39f in term 1.
I20260810 14:57:11.197157 19381 raft_consensus.cc:2781] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:11.197336 19381 raft_consensus.cc:686] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [term 1 LEADER]: Becoming Leader. State: Replica: 5dc4f925fbca4d4d884ba8849b7bd39f, State: Running, Role: LEADER
I20260810 14:57:11.197446 19381 consensus_queue.cc:237] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1812278958cb4c638d77d41c97ed6a3f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33933 } } peers { permanent_uuid: "5dc4f925fbca4d4d884ba8849b7bd39f" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 42927 } } peers { permanent_uuid: "d37696ed894042c39ea41a0e161ea8cb" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42151 } }
I20260810 14:57:11.198362 18829 catalog_manager.cc:5495] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5dc4f925fbca4d4d884ba8849b7bd39f (127.30.74.66). New cstate: current_term: 1 leader_uuid: "5dc4f925fbca4d4d884ba8849b7bd39f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1812278958cb4c638d77d41c97ed6a3f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dc4f925fbca4d4d884ba8849b7bd39f" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 42927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d37696ed894042c39ea41a0e161ea8cb" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42151 } health_report { overall_health: UNKNOWN } } }
W20260810 14:57:11.206652 19051 tablet.cc:2369] T 044b9098f6bc4cfeb5936ec09ed1b771 P 1812278958cb4c638d77d41c97ed6a3f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:57:11.226809 19360 tablet.cc:2369] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:57:11.238552 19371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:11.235875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53916 (local address 127.30.74.66:42927)
0810 14:57:11.235938 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:11.235940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:11.235953 (+ 13us) server_negotiation.cc:407] Connection header received
0810 14:57:11.236001 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:11.236002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:11.236023 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:11.236091 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:11.236421 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.236910 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.237830 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:11.238030 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:11.238096 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:11.238242 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:11.238333 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:11.238408 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:11.238460 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 14:57:11.238470 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 14:57:11.240700 18997 raft_consensus.cc:1261] T 044b9098f6bc4cfeb5936ec09ed1b771 P 1812278958cb4c638d77d41c97ed6a3f [term 1 FOLLOWER]: Refusing update from remote peer 5dc4f925fbca4d4d884ba8849b7bd39f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:57:11.240705 19307 raft_consensus.cc:1261] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb [term 1 FOLLOWER]: Refusing update from remote peer 5dc4f925fbca4d4d884ba8849b7bd39f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:57:11.241127 19381 consensus_queue.cc:1035] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1812278958cb4c638d77d41c97ed6a3f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 33933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:57:11.241302 19385 consensus_queue.cc:1035] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d37696ed894042c39ea41a0e161ea8cb" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 42151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:57:11.243748 19388 mvcc.cc:205] Tried to move back new op lower bound from 7316987212758851584 to 7316987212585488384. Current Snapshot: MvccSnapshot[applied={T|T < 7316987212758851584}]
I20260810 14:57:11.244107 19391 mvcc.cc:205] Tried to move back new op lower bound from 7316987212758851584 to 7316987212585488384. Current Snapshot: MvccSnapshot[applied={T|T < 7316987212758851584}]
I20260810 14:57:11.246961 18829 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:57:11.247056 18829 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:57:11.247865 18829 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 880c5f5848644d9e84ca502acf020545: Sending DeleteTablet for 3 replicas of tablet 044b9098f6bc4cfeb5936ec09ed1b771
I20260810 14:57:11.248203 18973 tablet_service.cc:1514] Processing DeleteTablet for tablet 044b9098f6bc4cfeb5936ec09ed1b771 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:11 UTC) from {username='slave'} at 127.0.0.1:35746
I20260810 14:57:11.248426 19282 tablet_service.cc:1514] Processing DeleteTablet for tablet 044b9098f6bc4cfeb5936ec09ed1b771 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:11 UTC) from {username='slave'} at 127.0.0.1:36102
I20260810 14:57:11.248533 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 18901
I20260810 14:57:11.248811 19400 tablet_replica.cc:330] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb: stopping tablet replica
I20260810 14:57:11.248920 19400 raft_consensus.cc:2229] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:57:11.249022 19400 raft_consensus.cc:2258] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:57:11.250252 19400 ts_tablet_manager.cc:1905] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:57:11.251595 19126 tablet_service.cc:1514] Processing DeleteTablet for tablet 044b9098f6bc4cfeb5936ec09ed1b771 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:11 UTC) from {username='slave'} at 127.0.0.1:53906
I20260810 14:57:11.251849 19401 tablet_replica.cc:330] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f: stopping tablet replica
I20260810 14:57:11.252027 19401 raft_consensus.cc:2229] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:57:11.252203 19401 raft_consensus.cc:2258] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:57:11.252339 19400 ts_tablet_manager.cc:1918] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:57:11.252528 19400 log.cc:1196] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/044b9098f6bc4cfeb5936ec09ed1b771
I20260810 14:57:11.252880 19400 ts_tablet_manager.cc:1939] T 044b9098f6bc4cfeb5936ec09ed1b771 P d37696ed894042c39ea41a0e161ea8cb: Deleting consensus metadata
I20260810 14:57:11.253136 19401 ts_tablet_manager.cc:1905] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:57:11.253384 18816 catalog_manager.cc:4841] TS d37696ed894042c39ea41a0e161ea8cb (127.30.74.67:42151): tablet 044b9098f6bc4cfeb5936ec09ed1b771 (table test-table [id=d9e8b959a43a4101bd205838d0bd9b91]) successfully deleted
I20260810 14:57:11.255949 19401 ts_tablet_manager.cc:1918] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:57:11.256085 19401 log.cc:1196] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/044b9098f6bc4cfeb5936ec09ed1b771
I20260810 14:57:11.256455 19401 ts_tablet_manager.cc:1939] T 044b9098f6bc4cfeb5936ec09ed1b771 P 5dc4f925fbca4d4d884ba8849b7bd39f: Deleting consensus metadata
I20260810 14:57:11.256922 18815 catalog_manager.cc:4841] TS 5dc4f925fbca4d4d884ba8849b7bd39f (127.30.74.66:42927): tablet 044b9098f6bc4cfeb5936ec09ed1b771 (table test-table [id=d9e8b959a43a4101bd205838d0bd9b91]) successfully deleted
W20260810 14:57:11.259845 18813 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:33933 (error 108)
I20260810 14:57:11.260418 19368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:11.260277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:33933 (local address 127.0.0.1:35760)
0810 14:57:11.260329 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:11.260376 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:33933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260810 14:57:11.260524 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 19053
W20260810 14:57:11.265576 18813 catalog_manager.cc:4584] TS 1812278958cb4c638d77d41c97ed6a3f (127.30.74.65:33933): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 044b9098f6bc4cfeb5936ec09ed1b771: Network error: Client connection negotiation failed: client connection to 127.30.74.65:33933: connect: Connection refused (error 111)
I20260810 14:57:11.273137 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 19210
I20260810 14:57:11.279776 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 18796
2026-08-10T14:57:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (904 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:11 dist-test-slave-0b58 krb5kdc[19405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:11 dist-test-slave-0b58 krb5kdc[19405](info): set up 2 sockets
Aug 10 14:57:11 dist-test-slave-0b58 krb5kdc[19405](info): commencing operation
krb5kdc: starting...
W20260810 14:57:12.384122 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:12 dist-test-slave-0b58 krb5kdc[19405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:57:12.519688 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:43271
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:35257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:43271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:12.605978 19421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:12.606287 19421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:12.606335 19421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:12.607645 19421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:12.607731 19421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:12.607756 19421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:12.607777 19421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:12.607805 19421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:12.609503 19421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:43271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:43271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:12.609835 19421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:12.610102 19421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:12.615335 19430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:12.615463 19431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:12.615386 19433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:12.615648 19421 server_base.cc:1034] running on GCE node
I20260810 14:57:12.615911 19421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:12.616225 19421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:12.617467 19421 hybrid_clock.cc:648] HybridClock initialized: now 1786373832617445 us; error 44 us; skew 500 ppm
Aug 10 14:57:12 dist-test-slave-0b58 krb5kdc[19405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373832, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:12.620128 19421 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:57:12.620723 19421 webserver.cc:460] Webserver started at http://127.30.74.126:43213/ using document root <none> and password file <none>
I20260810 14:57:12.620976 19421 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:12.621048 19421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:12.621218 19421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:12.622421 19421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "bdc13f8ad423478cbc51669b7be99d6e"
format_stamp: "Formatted at 2026-08-10 14:57:12 on dist-test-slave-0b58"
server_key: "86155ab9a8ca60539ea02a9669980afe"
server_key_iv: "bacaac88d58fc2feef31643662d89a08"
server_key_version: "encryptionkey@0"
I20260810 14:57:12.622840 19421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "bdc13f8ad423478cbc51669b7be99d6e"
format_stamp: "Formatted at 2026-08-10 14:57:12 on dist-test-slave-0b58"
server_key: "86155ab9a8ca60539ea02a9669980afe"
server_key_iv: "bacaac88d58fc2feef31643662d89a08"
server_key_version: "encryptionkey@0"
I20260810 14:57:12.624464 19421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:57:12.625422 19441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:12.625698 19421 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:57:12.625818 19421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "bdc13f8ad423478cbc51669b7be99d6e"
format_stamp: "Formatted at 2026-08-10 14:57:12 on dist-test-slave-0b58"
server_key: "86155ab9a8ca60539ea02a9669980afe"
server_key_iv: "bacaac88d58fc2feef31643662d89a08"
server_key_version: "encryptionkey@0"
I20260810 14:57:12.625917 19421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:12.637744 19421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:12.640484 19421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:12.640689 19421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:12.645032 19421 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:43271
I20260810 14:57:12.645069 19501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:43271 every 8 connection(s)
I20260810 14:57:12.645470 19421 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:12.646064 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 19421
I20260810 14:57:12.646234 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:12.646464 31017 external_mini_cluster.cc:1428] Setting key ac3f709382e04a79b48a00bc43b220d4
I20260810 14:57:12.646504 19502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:12.650164 19502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e: Bootstrap starting.
I20260810 14:57:12.650856 19502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:12.651176 19502 log.cc:824] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:12.651970 19502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e: No bootstrap required, opened a new log
Aug 10 14:57:12 dist-test-slave-0b58 krb5kdc[19405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:12.654428 19502 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc13f8ad423478cbc51669b7be99d6e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43271 } }
I20260810 14:57:12.654587 19502 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:12.654609 19502 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdc13f8ad423478cbc51669b7be99d6e, State: Initialized, Role: FOLLOWER
I20260810 14:57:12.654798 19502 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc13f8ad423478cbc51669b7be99d6e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43271 } }
I20260810 14:57:12.654899 19502 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:12.654937 19502 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:12.654995 19502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:12.655834 19502 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc13f8ad423478cbc51669b7be99d6e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43271 } }
I20260810 14:57:12.655967 19502 leader_election.cc:302] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdc13f8ad423478cbc51669b7be99d6e; no voters:
I20260810 14:57:12.656183 19502 leader_election.cc:288] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:12.656239 19508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:12.656445 19508 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [term 1 LEADER]: Becoming Leader. State: Replica: bdc13f8ad423478cbc51669b7be99d6e, State: Running, Role: LEADER
I20260810 14:57:12.656522 19502 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:12.656553 19508 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc13f8ad423478cbc51669b7be99d6e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43271 } }
I20260810 14:57:12.656930 19508 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdc13f8ad423478cbc51669b7be99d6e. Latest consensus state: current_term: 1 leader_uuid: "bdc13f8ad423478cbc51669b7be99d6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc13f8ad423478cbc51669b7be99d6e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43271 } } }
I20260810 14:57:12.656939 19510 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdc13f8ad423478cbc51669b7be99d6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdc13f8ad423478cbc51669b7be99d6e" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 43271 } } }
I20260810 14:57:12.657025 19508 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:12.657029 19510 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:12.657882 19516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:12.658843 19516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:12.659402 19506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:12.647076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47706 (local address 127.30.74.126:43271)
0810 14:57:12.647293 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:12.647304 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:12.647339 (+ 35us) server_negotiation.cc:407] Connection header received
0810 14:57:12.647785 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:12.647795 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:12.647860 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:12.648027 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:12.648747 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:12.650126 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:12.651019 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:12.651501 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:12.651621 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:12.654706 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:12.654710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:12.654710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:12.654725 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:12.656889 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:12.657418 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:12.657422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:12.657422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:12.657551 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:12.657805 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:12.657805 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:12.657806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:12.658173 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:12.658640 (+ 467us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:12.658826 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 14:57:12.659116 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":95,"threads_started":1}
I20260810 14:57:12.660686 19516 catalog_manager.cc:1347] Generated new cluster ID: ab3a31c49cb74989b01242b8d6a085eb
I20260810 14:57:12.660744 19516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:12.676957 19516 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:12.678048 19516 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:12.681557 19516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e: Generated new TSK 0
I20260810 14:57:12.681805 19516 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.74.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
Entry for principal kudu/127.30.74.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab.
I20260810 14:57:12.715323 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43271
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:35257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:12.803119 19531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:12.803359 19531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:12.803386 19531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:12.804718 19531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:12.804770 19531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:12.804800 19531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:57:12.806419 19531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.65.keytab
--principal=kudu/127.30.74.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:12.806702 19531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:12.806982 19531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:12.812592 19537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:12.812668 19541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:12.812661 19538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:12.812790 19531 server_base.cc:1034] running on GCE node
I20260810 14:57:12.813045 19531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:12.813366 19531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:12.814544 19531 hybrid_clock.cc:648] HybridClock initialized: now 1786373832814518 us; error 38 us; skew 500 ppm
Aug 10 14:57:12 dist-test-slave-0b58 krb5kdc[19405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373832, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:12.817443 19531 init.cc:377] Logged in from keytab as kudu/127.30.74.65@KRBTEST.COM (short username kudu)
I20260810 14:57:12.817996 19531 webserver.cc:460] Webserver started at http://127.30.74.65:33291/ using document root <none> and password file <none>
I20260810 14:57:12.818260 19531 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:12.818333 19531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:12.818491 19531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:12.819625 19531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "3a12ad3a37174158828eb51cc66f6f4f"
format_stamp: "Formatted at 2026-08-10 14:57:12 on dist-test-slave-0b58"
server_key: "4f8828b2ee49398b65ba73f7b6625819"
server_key_iv: "c775ee3ea0e4520505f4b4a02b14f973"
server_key_version: "encryptionkey@0"
I20260810 14:57:12.820020 19531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "3a12ad3a37174158828eb51cc66f6f4f"
format_stamp: "Formatted at 2026-08-10 14:57:12 on dist-test-slave-0b58"
server_key: "4f8828b2ee49398b65ba73f7b6625819"
server_key_iv: "c775ee3ea0e4520505f4b4a02b14f973"
server_key_version: "encryptionkey@0"
I20260810 14:57:12.821568 19531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:57:12.822449 19548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:12.822705 19531 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:12.822803 19531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "3a12ad3a37174158828eb51cc66f6f4f"
format_stamp: "Formatted at 2026-08-10 14:57:12 on dist-test-slave-0b58"
server_key: "4f8828b2ee49398b65ba73f7b6625819"
server_key_iv: "c775ee3ea0e4520505f4b4a02b14f973"
server_key_version: "encryptionkey@0"
I20260810 14:57:12.822896 19531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:12.836833 19531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:12.839486 19531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:12.839684 19531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:12.839962 19531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:12.840415 19531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:12.840473 19531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:12.840518 19531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:12.840547 19531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:12.846987 19531 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:39769
I20260810 14:57:12.847033 19681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:39769 every 8 connection(s)
I20260810 14:57:12.847389 19531 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:57:12.851956 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 19531
I20260810 14:57:12.852094 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:57:12.852444 31017 external_mini_cluster.cc:1428] Setting key 65a20298c46313a14f9059dd9c487233
Aug 10 14:57:12 dist-test-slave-0b58 krb5kdc[19405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373832, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.65@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:12.859551 19506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:12.848204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:38611 (local address 127.30.74.126:43271)
0810 14:57:12.848319 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:12.848321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:12.848618 (+ 297us) server_negotiation.cc:407] Connection header received
0810 14:57:12.849126 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:12.849128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:12.849141 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:12.849201 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:12.850258 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:12.850779 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:12.852245 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:12.852633 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:12.852728 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:12.855903 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:12.855905 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:12.855905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:12.855914 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:12.857907 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:12.858435 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:12.858438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:12.858438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:12.858520 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:12.858781 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:12.858781 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:12.858782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:12.858954 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:12.859036 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:12.859333 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 14:57:12.859441 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 14:57:12.859642 19684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:12.848206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43271 (local address 127.30.74.65:38611)
0810 14:57:12.848447 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:12.848498 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:12.848999 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:12.849285 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:12.849289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:12.849921 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:12.850137 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:12.850143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:12.851069 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:12.851070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:12.851990 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:12.851993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:12.852745 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:12.852747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:12.852954 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:12.853671 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:12.853681 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:12.855764 (+ 2083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:12.858016 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:12.858025 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:12.858027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:12.858360 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:12.858599 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:12.858599 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:12.858599 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:12.858712 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:12.859034 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:12.859041 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:12.859186 (+ 145us) client_negotiation.cc:769] Sending connection context
0810 14:57:12.859281 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 14:57:12.859435 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":146,"threads_started":1}
I20260810 14:57:12.860222 19682 heartbeater.cc:347] Connected to a master server at 127.30.74.126:43271
I20260810 14:57:12.860394 19682 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:12.860620 19682 heartbeater.cc:511] Master 127.30.74.126:43271 requested a full tablet report, sending...
I20260810 14:57:12.861274 19461 ts_manager.cc:194] Registered new tserver with Master: 3a12ad3a37174158828eb51cc66f6f4f (127.30.74.65:39769)
I20260810 14:57:12.862272 19461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.65@KRBTEST.COM'} at 127.30.74.65:38611
WARNING: no policy specified for kudu/127.30.74.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
Entry for principal kudu/127.30.74.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab.
I20260810 14:57:12.883052 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43271
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:35257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:12.971041 19688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:12.971278 19688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:12.971312 19688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:12.972581 19688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:12.972636 19688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:12.972666 19688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:57:12.974189 19688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.66.keytab
--principal=kudu/127.30.74.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:12.974431 19688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:12.974648 19688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:12.979789 19694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:12.979878 19696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:12.979971 19693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:12.980202 19688 server_base.cc:1034] running on GCE node
I20260810 14:57:12.980474 19688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:12.980701 19688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:12.981882 19688 hybrid_clock.cc:648] HybridClock initialized: now 1786373832981878 us; error 48 us; skew 500 ppm
Aug 10 14:57:12 dist-test-slave-0b58 krb5kdc[19405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373832, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:12.984817 19688 init.cc:377] Logged in from keytab as kudu/127.30.74.66@KRBTEST.COM (short username kudu)
I20260810 14:57:12.985303 19688 webserver.cc:460] Webserver started at http://127.30.74.66:38367/ using document root <none> and password file <none>
I20260810 14:57:12.985575 19688 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:12.985648 19688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:12.985816 19688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:12.986931 19688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "b46c9be4fdbd4068aae9b080569828ed"
format_stamp: "Formatted at 2026-08-10 14:57:12 on dist-test-slave-0b58"
server_key: "39875dfbae5ca8d6bb63de3089acf594"
server_key_iv: "1bd25fb976d3e0240542dde1eea735f8"
server_key_version: "encryptionkey@0"
I20260810 14:57:12.987318 19688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "b46c9be4fdbd4068aae9b080569828ed"
format_stamp: "Formatted at 2026-08-10 14:57:12 on dist-test-slave-0b58"
server_key: "39875dfbae5ca8d6bb63de3089acf594"
server_key_iv: "1bd25fb976d3e0240542dde1eea735f8"
server_key_version: "encryptionkey@0"
I20260810 14:57:12.988880 19688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:57:12.989763 19704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:12.989984 19688 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:12.990104 19688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "b46c9be4fdbd4068aae9b080569828ed"
format_stamp: "Formatted at 2026-08-10 14:57:12 on dist-test-slave-0b58"
server_key: "39875dfbae5ca8d6bb63de3089acf594"
server_key_iv: "1bd25fb976d3e0240542dde1eea735f8"
server_key_version: "encryptionkey@0"
I20260810 14:57:12.990196 19688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:13.003968 19688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:13.006893 19688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:13.007114 19688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:13.007395 19688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:13.007743 19688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:13.007782 19688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:13.007835 19688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:13.007864 19688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:13.014268 19688 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:40379
I20260810 14:57:13.014361 19836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:40379 every 8 connection(s)
I20260810 14:57:13.014693 19688 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:57:13.019860 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 19688
I20260810 14:57:13.020012 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:57:13.020291 31017 external_mini_cluster.cc:1428] Setting key 13ad77d1847682fc9149f41aa386dfbe
Aug 10 14:57:13 dist-test-slave-0b58 krb5kdc[19405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373832, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.66@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:13.026223 19506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.015545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:46573 (local address 127.30.74.126:43271)
0810 14:57:13.015632 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:13.015635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:13.015912 (+ 277us) server_negotiation.cc:407] Connection header received
0810 14:57:13.016607 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:13.016609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:13.016621 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:13.016683 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:13.017758 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.018310 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.019375 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.019568 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.019629 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.022590 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:13.022592 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:13.022592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:13.022601 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:13.024440 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:13.024897 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:13.024899 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:13.024899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:13.024967 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:13.025264 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:13.025265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:13.025266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:13.025402 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:13.025479 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:13.025962 (+ 483us) server_negotiation.cc:299] Negotiation successful
0810 14:57:13.026123 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 14:57:13.026382 19839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.015508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43271 (local address 127.30.74.66:46573)
0810 14:57:13.015707 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:13.015748 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:13.016462 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:13.016746 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:13.016750 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:13.017395 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:13.017612 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.017618 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.018425 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.018426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.019277 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.019280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.019682 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.019684 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.019865 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.020764 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:13.020776 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:13.022481 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:13.024535 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:13.024544 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:13.024547 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:13.024806 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:13.025053 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:13.025054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:13.025055 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:13.025175 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:13.025480 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:13.025487 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:13.025731 (+ 244us) client_negotiation.cc:769] Sending connection context
0810 14:57:13.025901 (+ 170us) client_negotiation.cc:240] Negotiation successful
0810 14:57:13.026129 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":114,"threads_started":1}
I20260810 14:57:13.027144 19837 heartbeater.cc:347] Connected to a master server at 127.30.74.126:43271
I20260810 14:57:13.027284 19837 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:13.027485 19837 heartbeater.cc:511] Master 127.30.74.126:43271 requested a full tablet report, sending...
I20260810 14:57:13.027992 19461 ts_manager.cc:194] Registered new tserver with Master: b46c9be4fdbd4068aae9b080569828ed (127.30.74.66:40379)
I20260810 14:57:13.028484 19461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.66@KRBTEST.COM'} at 127.30.74.66:46573
WARNING: no policy specified for kudu/127.30.74.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
Entry for principal kudu/127.30.74.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab.
I20260810 14:57:13.051421 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43271
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:35257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:13.144497 19843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:13.144730 19843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:13.144758 19843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:13.146027 19843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:13.146080 19843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:13.146124 19843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:57:13.147759 19843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.67.keytab
--principal=kudu/127.30.74.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:43271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:13.147989 19843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:13.148188 19843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:13.153753 19851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:13.153774 19843 server_base.cc:1034] running on GCE node
W20260810 14:57:13.153702 19849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:13.153733 19848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:13.154162 19843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:13.154445 19843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:13.155616 19843 hybrid_clock.cc:648] HybridClock initialized: now 1786373833155593 us; error 42 us; skew 500 ppm
Aug 10 14:57:13 dist-test-slave-0b58 krb5kdc[19405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373833, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:13.159165 19843 init.cc:377] Logged in from keytab as kudu/127.30.74.67@KRBTEST.COM (short username kudu)
I20260810 14:57:13.159781 19843 webserver.cc:460] Webserver started at http://127.30.74.67:39127/ using document root <none> and password file <none>
I20260810 14:57:13.160092 19843 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:13.160148 19843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:13.160326 19843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:13.161520 19843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "2b03b21cd5f34616ad17d7b26be0158c"
format_stamp: "Formatted at 2026-08-10 14:57:13 on dist-test-slave-0b58"
server_key: "645549be9e3de354c0d4229921443828"
server_key_iv: "7e138abd0c9650a4ff23310363e7b7f2"
server_key_version: "encryptionkey@0"
I20260810 14:57:13.161918 19843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "2b03b21cd5f34616ad17d7b26be0158c"
format_stamp: "Formatted at 2026-08-10 14:57:13 on dist-test-slave-0b58"
server_key: "645549be9e3de354c0d4229921443828"
server_key_iv: "7e138abd0c9650a4ff23310363e7b7f2"
server_key_version: "encryptionkey@0"
I20260810 14:57:13.163455 19843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:57:13.164458 19860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:13.164681 19843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:57:13.164810 19843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "2b03b21cd5f34616ad17d7b26be0158c"
format_stamp: "Formatted at 2026-08-10 14:57:13 on dist-test-slave-0b58"
server_key: "645549be9e3de354c0d4229921443828"
server_key_iv: "7e138abd0c9650a4ff23310363e7b7f2"
server_key_version: "encryptionkey@0"
I20260810 14:57:13.164911 19843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:13.191862 19843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:13.194486 19843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:13.194662 19843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:13.194928 19843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:13.195290 19843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:13.195348 19843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:13.195392 19843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:13.195423 19843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:13.201715 19843 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:38603
I20260810 14:57:13.201761 19988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:38603 every 8 connection(s)
I20260810 14:57:13.202150 19843 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:57:13.208568 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 19843
I20260810 14:57:13.208702 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
Aug 10 14:57:13 dist-test-slave-0b58 krb5kdc[19405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373833, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.67@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:13.208910 31017 external_mini_cluster.cc:1428] Setting key 4e7f6394b417c97eeafe08b30b6e1202
I20260810 14:57:13.212502 19506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.202998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:53705 (local address 127.30.74.126:43271)
0810 14:57:13.203075 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:13.203077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:13.203281 (+ 204us) server_negotiation.cc:407] Connection header received
0810 14:57:13.203855 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:13.203856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:13.203870 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:13.203923 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:13.205009 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.205521 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.206568 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.206773 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.206850 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.209541 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:13.209544 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:13.209544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:13.209553 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:13.211154 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:13.211570 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:13.211571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:13.211571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:13.211623 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:13.211846 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:13.211846 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:13.211847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:13.211967 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:13.212019 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:13.212362 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 14:57:13.212436 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 14:57:13.212745 19992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.202976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:43271 (local address 127.30.74.67:53705)
0810 14:57:13.203140 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:13.203183 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:13.203728 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:13.203997 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:13.204000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:13.204675 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:13.204884 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.204889 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.205641 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.205642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.206473 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.206475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.206873 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.206875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.207006 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.207532 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:13.207541 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:13.209399 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:13.211238 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:13.211247 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:13.211249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:13.211497 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:13.211681 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:13.211681 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:13.211682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:13.211788 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:13.212030 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:13.212037 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:13.212204 (+ 167us) client_negotiation.cc:769] Sending connection context
0810 14:57:13.212348 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 14:57:13.212496 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260810 14:57:13.213266 19990 heartbeater.cc:347] Connected to a master server at 127.30.74.126:43271
I20260810 14:57:13.213359 19990 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:13.213524 19990 heartbeater.cc:511] Master 127.30.74.126:43271 requested a full tablet report, sending...
I20260810 14:57:13.213856 19461 ts_manager.cc:194] Registered new tserver with Master: 2b03b21cd5f34616ad17d7b26be0158c (127.30.74.67:38603)
I20260810 14:57:13.214218 19461 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.74.67@KRBTEST.COM'} at 127.30.74.67:53705
I20260810 14:57:13.221171 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:57:13.229357 19506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.222723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47714 (local address 127.30.74.126:43271)
0810 14:57:13.222783 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:13.222785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:13.222841 (+ 56us) server_negotiation.cc:407] Connection header received
0810 14:57:13.222911 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:13.222912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:13.222922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:13.222974 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:13.223516 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.224041 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.224910 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.225125 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.225183 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.226531 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:13.226533 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:13.226533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:13.226540 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:13.228099 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:13.228550 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:13.228551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:13.228552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:13.228627 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:13.228860 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:13.228861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:13.228862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:13.229034 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:13.229116 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:13.229221 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 14:57:13.229285 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 14:57:13.231145 19461 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:57:13.232949 19461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:57:13.240497 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.235859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47016 (local address 127.30.74.67:38603)
0810 14:57:13.236378 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:13.236383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:13.236395 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:57:13.236437 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:13.236439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:13.236454 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:13.236532 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:13.236918 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.238003 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.239416 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.239959 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.240057 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.240129 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:13.240348 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 14:57:13.240398 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":101,"threads_started":1}
I20260810 14:57:13.240487 20001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.235677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:38603 (local address 127.0.0.1:47016)
0810 14:57:13.236055 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:13.236066 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:13.236108 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:13.236777 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:13.236778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:13.236779 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:13.236833 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.236835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.238108 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.238109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.239328 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.239330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.240059 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.240060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.240173 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.240189 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 14:57:13.240321 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 14:57:13.240399 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":105,"threads_started":1}
I20260810 14:57:13.240530 20003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.235766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51074 (local address 127.30.74.65:39769)
0810 14:57:13.235989 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:13.235995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:13.236007 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:57:13.236108 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:13.236111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:13.236130 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:13.236219 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:13.236423 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.237615 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.239263 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.239988 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.240064 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.240129 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:13.240390 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 14:57:13.240439 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":99,"threads_started":1}
I20260810 14:57:13.240530 20006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.236078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39498 (local address 127.30.74.66:40379)
0810 14:57:13.236494 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:13.236499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:13.236716 (+ 217us) server_negotiation.cc:407] Connection header received
0810 14:57:13.236772 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:13.236774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:13.236787 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:13.236880 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:13.237028 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.237822 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.239109 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.239984 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.240064 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.240131 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:13.240390 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 14:57:13.240436 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":107,"threads_started":1}
I20260810 14:57:13.241220 20002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.235622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:39769 (local address 127.0.0.1:51074)
0810 14:57:13.235846 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:13.235908 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:13.235982 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:13.236235 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:13.236239 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:13.236241 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:13.236339 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.236341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.237739 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.237740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.239166 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.239168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.240173 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.240174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.240248 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.240280 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 14:57:13.240335 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 14:57:13.241134 (+ 799us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":628,"thread_start_us":64,"threads_started":1}
I20260810 14:57:13.241434 20004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.235846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:40379 (local address 127.0.0.1:39498)
0810 14:57:13.236641 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:13.236652 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:13.236709 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:13.236891 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:13.236892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:13.236895 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:13.236952 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.236953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.237944 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.237945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.238991 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.238994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.240172 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.240173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.240248 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.240282 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 14:57:13.240335 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 14:57:13.241350 (+ 1015us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":665,"thread_start_us":65,"threads_started":1}
I20260810 14:57:13.241462 19915 tablet_service.cc:1467] Processing CreateTablet for tablet 54a460ff83d14c2682651c21f59041e8 (DEFAULT_TABLE table=test-table [id=8341d75c4ea048d5aa284dc9afe05fd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:13.241909 19915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54a460ff83d14c2682651c21f59041e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:13.242030 19607 tablet_service.cc:1467] Processing CreateTablet for tablet 54a460ff83d14c2682651c21f59041e8 (DEFAULT_TABLE table=test-table [id=8341d75c4ea048d5aa284dc9afe05fd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:13.242136 19762 tablet_service.cc:1467] Processing CreateTablet for tablet 54a460ff83d14c2682651c21f59041e8 (DEFAULT_TABLE table=test-table [id=8341d75c4ea048d5aa284dc9afe05fd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:13.242383 19607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54a460ff83d14c2682651c21f59041e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:13.242538 19762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54a460ff83d14c2682651c21f59041e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:13.245473 20007 tablet_bootstrap.cc:492] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed: Bootstrap starting.
I20260810 14:57:13.245996 20009 tablet_bootstrap.cc:492] T 54a460ff83d14c2682651c21f59041e8 P 3a12ad3a37174158828eb51cc66f6f4f: Bootstrap starting.
I20260810 14:57:13.246335 20007 tablet_bootstrap.cc:654] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:13.246686 20009 tablet_bootstrap.cc:654] T 54a460ff83d14c2682651c21f59041e8 P 3a12ad3a37174158828eb51cc66f6f4f: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:13.246928 20007 log.cc:824] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:13.247236 20009 log.cc:824] T 54a460ff83d14c2682651c21f59041e8 P 3a12ad3a37174158828eb51cc66f6f4f: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:13.247646 20008 tablet_bootstrap.cc:492] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c: Bootstrap starting.
I20260810 14:57:13.247721 20007 tablet_bootstrap.cc:492] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed: No bootstrap required, opened a new log
I20260810 14:57:13.247886 20007 ts_tablet_manager.cc:1397] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:57:13.248121 20009 tablet_bootstrap.cc:492] T 54a460ff83d14c2682651c21f59041e8 P 3a12ad3a37174158828eb51cc66f6f4f: No bootstrap required, opened a new log
I20260810 14:57:13.248193 20009 ts_tablet_manager.cc:1397] T 54a460ff83d14c2682651c21f59041e8 P 3a12ad3a37174158828eb51cc66f6f4f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:57:13.248389 20008 tablet_bootstrap.cc:654] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:13.248908 20008 log.cc:824] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:13.249681 20008 tablet_bootstrap.cc:492] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c: No bootstrap required, opened a new log
I20260810 14:57:13.249763 20008 ts_tablet_manager.cc:1397] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 14:57:13.250088 20007 raft_consensus.cc:348] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a12ad3a37174158828eb51cc66f6f4f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 39769 } } peers { permanent_uuid: "b46c9be4fdbd4068aae9b080569828ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 40379 } } peers { permanent_uuid: "2b03b21cd5f34616ad17d7b26be0158c" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38603 } }
I20260810 14:57:13.250305 20007 raft_consensus.cc:374] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:13.250355 20007 raft_consensus.cc:729] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b46c9be4fdbd4068aae9b080569828ed, State: Initialized, Role: FOLLOWER
I20260810 14:57:13.250267 20009 raft_consensus.cc:348] T 54a460ff83d14c2682651c21f59041e8 P 3a12ad3a37174158828eb51cc66f6f4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a12ad3a37174158828eb51cc66f6f4f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 39769 } } peers { permanent_uuid: "b46c9be4fdbd4068aae9b080569828ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 40379 } } peers { permanent_uuid: "2b03b21cd5f34616ad17d7b26be0158c" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38603 } }
I20260810 14:57:13.250427 20009 raft_consensus.cc:374] T 54a460ff83d14c2682651c21f59041e8 P 3a12ad3a37174158828eb51cc66f6f4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:13.250465 20009 raft_consensus.cc:729] T 54a460ff83d14c2682651c21f59041e8 P 3a12ad3a37174158828eb51cc66f6f4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a12ad3a37174158828eb51cc66f6f4f, State: Initialized, Role: FOLLOWER
I20260810 14:57:13.250516 20007 consensus_queue.cc:260] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a12ad3a37174158828eb51cc66f6f4f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 39769 } } peers { permanent_uuid: "b46c9be4fdbd4068aae9b080569828ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 40379 } } peers { permanent_uuid: "2b03b21cd5f34616ad17d7b26be0158c" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38603 } }
I20260810 14:57:13.250559 20009 consensus_queue.cc:260] T 54a460ff83d14c2682651c21f59041e8 P 3a12ad3a37174158828eb51cc66f6f4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a12ad3a37174158828eb51cc66f6f4f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 39769 } } peers { permanent_uuid: "b46c9be4fdbd4068aae9b080569828ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 40379 } } peers { permanent_uuid: "2b03b21cd5f34616ad17d7b26be0158c" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38603 } }
I20260810 14:57:13.250856 20009 ts_tablet_manager.cc:1428] T 54a460ff83d14c2682651c21f59041e8 P 3a12ad3a37174158828eb51cc66f6f4f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 14:57:13.250856 20007 ts_tablet_manager.cc:1428] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:57:13.251076 19682 heartbeater.cc:503] Master 127.30.74.126:43271 was elected leader, sending a full tablet report...
I20260810 14:57:13.251068 19837 heartbeater.cc:503] Master 127.30.74.126:43271 was elected leader, sending a full tablet report...
I20260810 14:57:13.251955 20008 raft_consensus.cc:348] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a12ad3a37174158828eb51cc66f6f4f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 39769 } } peers { permanent_uuid: "b46c9be4fdbd4068aae9b080569828ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 40379 } } peers { permanent_uuid: "2b03b21cd5f34616ad17d7b26be0158c" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38603 } }
I20260810 14:57:13.252173 20008 raft_consensus.cc:374] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:13.252224 20008 raft_consensus.cc:729] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b03b21cd5f34616ad17d7b26be0158c, State: Initialized, Role: FOLLOWER
I20260810 14:57:13.252368 20008 consensus_queue.cc:260] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a12ad3a37174158828eb51cc66f6f4f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 39769 } } peers { permanent_uuid: "b46c9be4fdbd4068aae9b080569828ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 40379 } } peers { permanent_uuid: "2b03b21cd5f34616ad17d7b26be0158c" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38603 } }
I20260810 14:57:13.252679 20008 ts_tablet_manager.cc:1428] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 14:57:13.252767 19990 heartbeater.cc:503] Master 127.30.74.126:43271 was elected leader, sending a full tablet report...
W20260810 14:57:13.265347 19838 tablet.cc:2369] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:57:13.293931 20015 raft_consensus.cc:482] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:57:13.294075 20015 raft_consensus.cc:504] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a12ad3a37174158828eb51cc66f6f4f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 39769 } } peers { permanent_uuid: "b46c9be4fdbd4068aae9b080569828ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 40379 } } peers { permanent_uuid: "2b03b21cd5f34616ad17d7b26be0158c" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38603 } }
I20260810 14:57:13.294481 20015 leader_election.cc:288] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a12ad3a37174158828eb51cc66f6f4f (127.30.74.65:39769), b46c9be4fdbd4068aae9b080569828ed (127.30.74.66:40379)
I20260810 14:57:13.298035 19992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.294821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:39769 (local address 127.30.74.67:49807)
0810 14:57:13.294898 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:13.294912 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:13.294961 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:13.295214 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:13.295216 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:13.295219 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:13.295330 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.295332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.296064 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.296065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.297151 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.297154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.297771 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.297773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.297865 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.297869 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:57:13.297924 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 14:57:13.297973 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 14:57:13.298187 20003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.294912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:49807 (local address 127.30.74.65:39769)
0810 14:57:13.294984 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:13.294986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:13.294999 (+ 13us) server_negotiation.cc:407] Connection header received
0810 14:57:13.295036 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:13.295037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:13.295048 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:13.295148 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:13.295428 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.295973 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.297229 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.297682 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.297745 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.297753 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:13.298047 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 14:57:13.298095 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 14:57:13.298305 20006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.295074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:57003 (local address 127.30.74.66:40379)
0810 14:57:13.295141 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:13.295185 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:13.295216 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:57:13.295303 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:13.295304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:13.295316 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:13.295381 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:13.295563 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.296078 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.297357 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.297776 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.297851 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.297860 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:13.298171 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 14:57:13.298223 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 14:57:13.298525 20016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.294969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:40379 (local address 127.30.74.67:57003)
0810 14:57:13.295162 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:13.295171 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:13.295229 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:13.295414 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:13.295415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:13.295418 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:13.295492 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.295494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.296169 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.296170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.297269 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.297271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.297876 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.297877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.298068 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.298073 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:57:13.298404 (+ 331us) client_negotiation.cc:240] Negotiation successful
0810 14:57:13.298457 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":130,"threads_started":1}
I20260810 14:57:13.298522 19630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a460ff83d14c2682651c21f59041e8" candidate_uuid: "2b03b21cd5f34616ad17d7b26be0158c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a12ad3a37174158828eb51cc66f6f4f" is_pre_election: true
I20260810 14:57:13.298727 19630 raft_consensus.cc:2454] T 54a460ff83d14c2682651c21f59041e8 P 3a12ad3a37174158828eb51cc66f6f4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b03b21cd5f34616ad17d7b26be0158c in term 0.
I20260810 14:57:13.298918 19782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a460ff83d14c2682651c21f59041e8" candidate_uuid: "2b03b21cd5f34616ad17d7b26be0158c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b46c9be4fdbd4068aae9b080569828ed" is_pre_election: true
I20260810 14:57:13.298995 19864 leader_election.cc:302] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b03b21cd5f34616ad17d7b26be0158c, 3a12ad3a37174158828eb51cc66f6f4f; no voters:
I20260810 14:57:13.299114 19782 raft_consensus.cc:2454] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b03b21cd5f34616ad17d7b26be0158c in term 0.
I20260810 14:57:13.299180 20015 raft_consensus.cc:2781] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:57:13.299266 20015 raft_consensus.cc:482] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:57:13.299305 20015 raft_consensus.cc:3037] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:13.300074 20015 raft_consensus.cc:504] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a12ad3a37174158828eb51cc66f6f4f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 39769 } } peers { permanent_uuid: "b46c9be4fdbd4068aae9b080569828ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 40379 } } peers { permanent_uuid: "2b03b21cd5f34616ad17d7b26be0158c" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38603 } }
I20260810 14:57:13.300230 20015 leader_election.cc:288] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [CANDIDATE]: Term 1 election: Requested vote from peers 3a12ad3a37174158828eb51cc66f6f4f (127.30.74.65:39769), b46c9be4fdbd4068aae9b080569828ed (127.30.74.66:40379)
I20260810 14:57:13.300473 19630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a460ff83d14c2682651c21f59041e8" candidate_uuid: "2b03b21cd5f34616ad17d7b26be0158c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a12ad3a37174158828eb51cc66f6f4f"
I20260810 14:57:13.300518 19782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54a460ff83d14c2682651c21f59041e8" candidate_uuid: "2b03b21cd5f34616ad17d7b26be0158c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b46c9be4fdbd4068aae9b080569828ed"
I20260810 14:57:13.300577 19630 raft_consensus.cc:3037] T 54a460ff83d14c2682651c21f59041e8 P 3a12ad3a37174158828eb51cc66f6f4f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:13.300614 19782 raft_consensus.cc:3037] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:13.301393 19782 raft_consensus.cc:2454] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b03b21cd5f34616ad17d7b26be0158c in term 1.
I20260810 14:57:13.301414 19630 raft_consensus.cc:2454] T 54a460ff83d14c2682651c21f59041e8 P 3a12ad3a37174158828eb51cc66f6f4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b03b21cd5f34616ad17d7b26be0158c in term 1.
I20260810 14:57:13.301607 19864 leader_election.cc:302] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b03b21cd5f34616ad17d7b26be0158c, b46c9be4fdbd4068aae9b080569828ed; no voters:
I20260810 14:57:13.301733 20015 raft_consensus.cc:2781] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:13.301892 20015 raft_consensus.cc:686] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [term 1 LEADER]: Becoming Leader. State: Replica: 2b03b21cd5f34616ad17d7b26be0158c, State: Running, Role: LEADER
I20260810 14:57:13.302011 20015 consensus_queue.cc:237] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a12ad3a37174158828eb51cc66f6f4f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 39769 } } peers { permanent_uuid: "b46c9be4fdbd4068aae9b080569828ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 40379 } } peers { permanent_uuid: "2b03b21cd5f34616ad17d7b26be0158c" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38603 } }
I20260810 14:57:13.302994 19460 catalog_manager.cc:5495] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b03b21cd5f34616ad17d7b26be0158c (127.30.74.67). New cstate: current_term: 1 leader_uuid: "2b03b21cd5f34616ad17d7b26be0158c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a12ad3a37174158828eb51cc66f6f4f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 39769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b46c9be4fdbd4068aae9b080569828ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 40379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b03b21cd5f34616ad17d7b26be0158c" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 38603 } health_report { overall_health: HEALTHY } } }
I20260810 14:57:13.317015 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.314426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47024 (local address 127.30.74.67:38603)
0810 14:57:13.314497 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:13.314499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:13.314508 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:57:13.314556 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:13.314557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:13.314581 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:13.314668 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:13.314832 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.315318 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.316127 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.316384 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.316459 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.316658 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:13.316778 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:13.316872 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:13.316898 (+ 26us) server_negotiation.cc:299] Negotiation successful
0810 14:57:13.316920 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 14:57:13.318994 19782 raft_consensus.cc:1261] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed [term 1 FOLLOWER]: Refusing update from remote peer 2b03b21cd5f34616ad17d7b26be0158c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:57:13.319036 19630 raft_consensus.cc:1261] T 54a460ff83d14c2682651c21f59041e8 P 3a12ad3a37174158828eb51cc66f6f4f [term 1 FOLLOWER]: Refusing update from remote peer 2b03b21cd5f34616ad17d7b26be0158c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:57:13.319454 20015 consensus_queue.cc:1035] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b46c9be4fdbd4068aae9b080569828ed" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 40379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:57:13.319610 20017 consensus_queue.cc:1035] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a12ad3a37174158828eb51cc66f6f4f" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 39769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:57:13.322703 20022 mvcc.cc:205] Tried to move back new op lower bound from 7316987221271764992 to 7316987221205762048. Current Snapshot: MvccSnapshot[applied={T|T < 7316987221271764992}]
I20260810 14:57:13.322928 20023 mvcc.cc:205] Tried to move back new op lower bound from 7316987221271764992 to 7316987221205762048. Current Snapshot: MvccSnapshot[applied={T|T < 7316987221271764992}]
I20260810 14:57:13.325538 19461 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:57:13.325635 19461 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:57:13.326489 19461 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bdc13f8ad423478cbc51669b7be99d6e: Sending DeleteTablet for 3 replicas of tablet 54a460ff83d14c2682651c21f59041e8
I20260810 14:57:13.326900 19607 tablet_service.cc:1514] Processing DeleteTablet for tablet 54a460ff83d14c2682651c21f59041e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:13 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:51074
I20260810 14:57:13.326898 19915 tablet_service.cc:1514] Processing DeleteTablet for tablet 54a460ff83d14c2682651c21f59041e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:13 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:47016
I20260810 14:57:13.327140 19762 tablet_service.cc:1514] Processing DeleteTablet for tablet 54a460ff83d14c2682651c21f59041e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:13 UTC) from {username='kudu', principal='kudu/127.30.74.126@KRBTEST.COM'} at 127.0.0.1:39498
I20260810 14:57:13.327431 20034 tablet_replica.cc:330] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed: stopping tablet replica
I20260810 14:57:13.327595 20034 raft_consensus.cc:2229] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:57:13.327684 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 19531
I20260810 14:57:13.331375 20032 tablet_replica.cc:330] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c: stopping tablet replica
I20260810 14:57:13.331523 20032 raft_consensus.cc:2229] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:57:13.331696 20032 raft_consensus.cc:2258] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:57:13.327767 20034 raft_consensus.cc:2258] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:57:13.333103 20034 ts_tablet_manager.cc:1905] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:57:13.335287 20034 ts_tablet_manager.cc:1918] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:57:13.335386 20034 log.cc:1196] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/54a460ff83d14c2682651c21f59041e8
I20260810 14:57:13.335703 20034 ts_tablet_manager.cc:1939] T 54a460ff83d14c2682651c21f59041e8 P b46c9be4fdbd4068aae9b080569828ed: Deleting consensus metadata
I20260810 14:57:13.336423 19444 catalog_manager.cc:4841] TS b46c9be4fdbd4068aae9b080569828ed (127.30.74.66:40379): tablet 54a460ff83d14c2682651c21f59041e8 (table test-table [id=8341d75c4ea048d5aa284dc9afe05fd4]) successfully deleted
W20260810 14:57:13.336898 19444 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:39769 (error 108)
I20260810 14:57:13.337224 20032 ts_tablet_manager.cc:1905] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:57:13.337517 20004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.337354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:39769 (local address 127.0.0.1:51076)
0810 14:57:13.337433 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:13.337471 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:39769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 14:57:13.337597 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 19688
I20260810 14:57:13.345562 20032 ts_tablet_manager.cc:1918] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:57:13.345712 20032 log.cc:1196] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/54a460ff83d14c2682651c21f59041e8
I20260810 14:57:13.346058 20032 ts_tablet_manager.cc:1939] T 54a460ff83d14c2682651c21f59041e8 P 2b03b21cd5f34616ad17d7b26be0158c: Deleting consensus metadata
W20260810 14:57:13.346319 19444 catalog_manager.cc:4584] TS 3a12ad3a37174158828eb51cc66f6f4f (127.30.74.65:39769): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 54a460ff83d14c2682651c21f59041e8: Network error: Client connection negotiation failed: client connection to 127.30.74.65:39769: connect: Connection refused (error 111)
I20260810 14:57:13.346725 19445 catalog_manager.cc:4841] TS 2b03b21cd5f34616ad17d7b26be0158c (127.30.74.67:38603): tablet 54a460ff83d14c2682651c21f59041e8 (table test-table [id=8341d75c4ea048d5aa284dc9afe05fd4]) successfully deleted
I20260810 14:57:13.350325 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 19843
I20260810 14:57:13.358475 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 19421
2026-08-10T14:57:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2081 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T14:57:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:13Z Disabled control of system clock
I20260810 14:57:13.392117 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:39827
--webserver_interface=127.30.74.126
--webserver_port=0
--builtin_ntp_servers=127.30.74.84:35515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:39827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:57:13.478567 20040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:13.478830 20040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:13.478854 20040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:13.480047 20040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:13.480090 20040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:13.480103 20040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:13.480116 20040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:13.480129 20040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:13.481580 20040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:39827
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:39827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:13.481827 20040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:13.482100 20040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:13.487293 20048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:13.487293 20050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:13.487295 20047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:13.487581 20040 server_base.cc:1034] running on GCE node
I20260810 14:57:13.487845 20040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:13.488176 20040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:13.489353 20040 hybrid_clock.cc:648] HybridClock initialized: now 1786373833489337 us; error 42 us; skew 500 ppm
I20260810 14:57:13.490813 20040 webserver.cc:460] Webserver started at http://127.30.74.126:45573/ using document root <none> and password file <none>
I20260810 14:57:13.491080 20040 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:13.491181 20040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:13.491338 20040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:13.492594 20040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "f1bf640882974123b2a0e8a2d818399a"
format_stamp: "Formatted at 2026-08-10 14:57:13 on dist-test-slave-0b58"
server_key: "d107b233c94e38c9d173f31f8e4cf8dd"
server_key_iv: "6f4547d30619354ad2206be79dfa4edd"
server_key_version: "encryptionkey@0"
I20260810 14:57:13.492982 20040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "f1bf640882974123b2a0e8a2d818399a"
format_stamp: "Formatted at 2026-08-10 14:57:13 on dist-test-slave-0b58"
server_key: "d107b233c94e38c9d173f31f8e4cf8dd"
server_key_iv: "6f4547d30619354ad2206be79dfa4edd"
server_key_version: "encryptionkey@0"
I20260810 14:57:13.494424 20040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:57:13.495343 20056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:13.495579 20040 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:13.495690 20040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "f1bf640882974123b2a0e8a2d818399a"
format_stamp: "Formatted at 2026-08-10 14:57:13 on dist-test-slave-0b58"
server_key: "d107b233c94e38c9d173f31f8e4cf8dd"
server_key_iv: "6f4547d30619354ad2206be79dfa4edd"
server_key_version: "encryptionkey@0"
I20260810 14:57:13.495774 20040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:13.523042 20040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:13.523427 20040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:13.523586 20040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:13.528211 20040 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:39827
I20260810 14:57:13.528277 20115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:39827 every 8 connection(s)
I20260810 14:57:13.528697 20040 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:13.529758 20116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:13.532143 20116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a: Bootstrap starting.
I20260810 14:57:13.532856 20116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:13.533128 20116 log.cc:824] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:13.533921 20116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a: No bootstrap required, opened a new log
I20260810 14:57:13.535828 20116 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bf640882974123b2a0e8a2d818399a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39827 } }
I20260810 14:57:13.536005 20116 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:13.536029 20116 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bf640882974123b2a0e8a2d818399a, State: Initialized, Role: FOLLOWER
I20260810 14:57:13.536149 20116 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bf640882974123b2a0e8a2d818399a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39827 } }
I20260810 14:57:13.536293 20116 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:13.536370 20116 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:13.536441 20116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:13.537123 20116 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bf640882974123b2a0e8a2d818399a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39827 } }
I20260810 14:57:13.537237 20116 leader_election.cc:302] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1bf640882974123b2a0e8a2d818399a; no voters:
I20260810 14:57:13.537456 20116 leader_election.cc:288] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:13.537896 20119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:13.538175 20116 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:13.538219 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 20040
I20260810 14:57:13.538362 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:13.538576 31017 external_mini_cluster.cc:1428] Setting key fb2d9819e36412e3fb59d935a466d2f7
I20260810 14:57:13.538964 20119 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [term 1 LEADER]: Becoming Leader. State: Replica: f1bf640882974123b2a0e8a2d818399a, State: Running, Role: LEADER
I20260810 14:57:13.539089 20119 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bf640882974123b2a0e8a2d818399a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39827 } }
I20260810 14:57:13.539701 20126 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1bf640882974123b2a0e8a2d818399a. Latest consensus state: current_term: 1 leader_uuid: "f1bf640882974123b2a0e8a2d818399a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bf640882974123b2a0e8a2d818399a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39827 } } }
I20260810 14:57:13.539798 20126 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:13.539918 20120 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1bf640882974123b2a0e8a2d818399a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1bf640882974123b2a0e8a2d818399a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 39827 } } }
I20260810 14:57:13.540004 20120 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:13.540584 20138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:13.541157 20138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:13.542919 20138 catalog_manager.cc:1347] Generated new cluster ID: 813a058b53124c52a9d2aa8a5b41c138
I20260810 14:57:13.542976 20138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:13.545856 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.539509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41532 (local address 127.30.74.126:39827)
0810 14:57:13.540089 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:13.540098 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:13.540134 (+ 36us) server_negotiation.cc:407] Connection header received
0810 14:57:13.540305 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:13.540314 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:13.540398 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:13.540578 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:13.540874 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.542232 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.543516 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.544120 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.544248 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.545100 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:13.545103 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:13.545104 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:13.545318 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:13.545364 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:13.545378 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:13.545446 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:13.545556 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 14:57:13.545640 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
I20260810 14:57:13.564090 20138 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:13.565021 20138 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:13.568177 20138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a: Generated new TSK 0
I20260810 14:57:13.568465 20138 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:13.574255 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.65:0
--local_ip_for_outbound_sockets=127.30.74.65
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39827
--builtin_ntp_servers=127.30.74.84:35515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:57:13.663317 20144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:13.663604 20144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:13.663645 20144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:13.664961 20144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:13.665037 20144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:13.665083 20144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.65
I20260810 14:57:13.666661 20144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.74.65
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.65
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:13.666976 20144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:13.667480 20144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:13.673812 20149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:13.673812 20150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:13.673846 20152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:13.674554 20144 server_base.cc:1034] running on GCE node
I20260810 14:57:13.674826 20144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:13.675102 20144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:13.676335 20144 hybrid_clock.cc:648] HybridClock initialized: now 1786373833676310 us; error 39 us; skew 500 ppm
I20260810 14:57:13.677834 20144 webserver.cc:460] Webserver started at http://127.30.74.65:46377/ using document root <none> and password file <none>
I20260810 14:57:13.678119 20144 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:13.678198 20144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:13.678416 20144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:13.679562 20144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/data/instance:
uuid: "c353cc3825dc47099dea090f908f063c"
format_stamp: "Formatted at 2026-08-10 14:57:13 on dist-test-slave-0b58"
server_key: "afee7737afe36fd10e88f35929fb5cfd"
server_key_iv: "0295553fc3b99570162b6fc9ca3be0fc"
server_key_version: "encryptionkey@0"
I20260810 14:57:13.679967 20144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance:
uuid: "c353cc3825dc47099dea090f908f063c"
format_stamp: "Formatted at 2026-08-10 14:57:13 on dist-test-slave-0b58"
server_key: "afee7737afe36fd10e88f35929fb5cfd"
server_key_iv: "0295553fc3b99570162b6fc9ca3be0fc"
server_key_version: "encryptionkey@0"
I20260810 14:57:13.681553 20144 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:57:13.682511 20158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:13.682761 20144 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:13.682883 20144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/wal
uuid: "c353cc3825dc47099dea090f908f063c"
format_stamp: "Formatted at 2026-08-10 14:57:13 on dist-test-slave-0b58"
server_key: "afee7737afe36fd10e88f35929fb5cfd"
server_key_iv: "0295553fc3b99570162b6fc9ca3be0fc"
server_key_version: "encryptionkey@0"
I20260810 14:57:13.682967 20144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:13.699146 20144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:13.699558 20144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:13.699731 20144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:13.700073 20144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:13.700541 20144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:13.700595 20144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:13.700627 20144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:13.700655 20144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:13.708761 20144 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.65:37089
I20260810 14:57:13.708819 20292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.65:37089 every 8 connection(s)
I20260810 14:57:13.709213 20144 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/data/info.pb
I20260810 14:57:13.710779 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 20144
I20260810 14:57:13.710942 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-0/wal/instance
I20260810 14:57:13.711166 31017 external_mini_cluster.cc:1428] Setting key 85c45d1d85c945fb24a2d97303d176d7
I20260810 14:57:13.713125 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.66:0
--local_ip_for_outbound_sockets=127.30.74.66
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39827
--builtin_ntp_servers=127.30.74.84:35515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:57:13.716388 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.710126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.65:42801 (local address 127.30.74.126:39827)
0810 14:57:13.710385 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:13.710388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:13.710400 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:57:13.711325 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:13.711326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:13.711346 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:13.711442 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:13.712479 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.713443 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.714557 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.714781 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.714866 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.715996 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:13.715998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:13.715998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:13.716005 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:13.716023 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:13.716028 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:13.716106 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:13.716218 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 14:57:13.716301 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230}
I20260810 14:57:13.716624 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.710071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39827 (local address 127.30.74.65:42801)
0810 14:57:13.710284 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:13.710329 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:13.711149 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:13.711526 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:13.711531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:13.711543 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:13.712340 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.712348 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.713571 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.713573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.714452 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.714455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.714876 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.714877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.715040 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.715792 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:57:13.715793 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:13.715826 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:57:13.715854 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:13.716109 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:13.716115 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:13.716119 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:57:13.716221 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 14:57:13.716368 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260810 14:57:13.717162 20293 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39827
I20260810 14:57:13.717268 20293 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:13.717489 20293 heartbeater.cc:511] Master 127.30.74.126:39827 requested a full tablet report, sending...
I20260810 14:57:13.718072 20075 ts_manager.cc:194] Registered new tserver with Master: c353cc3825dc47099dea090f908f063c (127.30.74.65:37089)
I20260810 14:57:13.718730 20075 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.74.65:42801
W20260810 14:57:13.803573 20297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:13.803835 20297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:13.803874 20297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:13.805109 20297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:13.805183 20297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:13.805227 20297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.66
I20260810 14:57:13.806690 20297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.74.66
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.66
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:13.806990 20297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:13.807336 20297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:13.812857 20305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:13.812865 20307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:13.812857 20304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:13.813349 20297 server_base.cc:1034] running on GCE node
I20260810 14:57:13.813614 20297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:13.813898 20297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:13.815102 20297 hybrid_clock.cc:648] HybridClock initialized: now 1786373833815079 us; error 45 us; skew 500 ppm
I20260810 14:57:13.816641 20297 webserver.cc:460] Webserver started at http://127.30.74.66:38125/ using document root <none> and password file <none>
I20260810 14:57:13.816931 20297 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:13.816983 20297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:13.817093 20297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:13.818222 20297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/data/instance:
uuid: "40c0a3915e224f0698a76912165d871b"
format_stamp: "Formatted at 2026-08-10 14:57:13 on dist-test-slave-0b58"
server_key: "5fdd5d023f13bd43a69e55c0ba509809"
server_key_iv: "511f1d4a04a3576a44c89fd2f9ad5c7e"
server_key_version: "encryptionkey@0"
I20260810 14:57:13.818553 20297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance:
uuid: "40c0a3915e224f0698a76912165d871b"
format_stamp: "Formatted at 2026-08-10 14:57:13 on dist-test-slave-0b58"
server_key: "5fdd5d023f13bd43a69e55c0ba509809"
server_key_iv: "511f1d4a04a3576a44c89fd2f9ad5c7e"
server_key_version: "encryptionkey@0"
I20260810 14:57:13.819958 20297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:57:13.820868 20313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:13.821065 20297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 14:57:13.821161 20297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/wal
uuid: "40c0a3915e224f0698a76912165d871b"
format_stamp: "Formatted at 2026-08-10 14:57:13 on dist-test-slave-0b58"
server_key: "5fdd5d023f13bd43a69e55c0ba509809"
server_key_iv: "511f1d4a04a3576a44c89fd2f9ad5c7e"
server_key_version: "encryptionkey@0"
I20260810 14:57:13.821224 20297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:13.831873 20297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:13.832182 20297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:13.832398 20297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:13.832620 20297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:13.832922 20297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:13.832952 20297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:13.832975 20297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:13.832988 20297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:13.840119 20297 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.66:35375
I20260810 14:57:13.840247 20438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.66:35375 every 8 connection(s)
I20260810 14:57:13.840637 20297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/data/info.pb
I20260810 14:57:13.846462 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.841546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:34051 (local address 127.30.74.126:39827)
0810 14:57:13.841620 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:13.841622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:13.841898 (+ 276us) server_negotiation.cc:407] Connection header received
0810 14:57:13.842398 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:13.842400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:13.842411 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:13.842483 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:13.843242 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.843778 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.844909 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.845102 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.845166 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.846181 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:13.846183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:13.846183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:13.846189 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:13.846207 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:13.846212 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:13.846267 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:13.846359 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 14:57:13.846391 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 14:57:13.846726 20443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.841528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39827 (local address 127.30.74.66:34051)
0810 14:57:13.841751 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:13.841795 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:13.842302 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:13.842544 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:13.842548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:13.842557 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:13.843133 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.843139 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.843880 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.843880 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.844812 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.844815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.845187 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.845188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.845314 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.845961 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:57:13.845961 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:13.845993 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:57:13.846020 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:13.846279 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:13.846284 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:13.846287 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:57:13.846345 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 14:57:13.846443 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":112,"threads_started":1}
I20260810 14:57:13.847750 20439 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39827
I20260810 14:57:13.847925 20439 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:13.848176 20439 heartbeater.cc:511] Master 127.30.74.126:39827 requested a full tablet report, sending...
I20260810 14:57:13.848762 20075 ts_manager.cc:194] Registered new tserver with Master: 40c0a3915e224f0698a76912165d871b (127.30.74.66:35375)
I20260810 14:57:13.849185 20075 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.74.66:34051
I20260810 14:57:13.850272 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 20297
I20260810 14:57:13.850407 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/wal/instance
I20260810 14:57:13.850625 31017 external_mini_cluster.cc:1428] Setting key 75f77728153997698cb47fea907ab223
I20260810 14:57:13.851830 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.74.67:0
--local_ip_for_outbound_sockets=127.30.74.67
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39827
--builtin_ntp_servers=127.30.74.84:35515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:57:13.940582 20444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:13.940862 20444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:13.940888 20444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:13.942350 20444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:13.942412 20444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:13.942442 20444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.74.67
I20260810 14:57:13.944049 20444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.74.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.74.67
--webserver_port=0
--tserver_master_addrs=127.30.74.126:39827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.74.67
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:13.944411 20444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:13.944787 20444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:13.950618 20452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:13.950591 20449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:13.950610 20444 server_base.cc:1034] running on GCE node
W20260810 14:57:13.950580 20450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:13.951119 20444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:13.951448 20444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:13.952634 20444 hybrid_clock.cc:648] HybridClock initialized: now 1786373833952620 us; error 42 us; skew 500 ppm
I20260810 14:57:13.954173 20444 webserver.cc:460] Webserver started at http://127.30.74.67:35801/ using document root <none> and password file <none>
I20260810 14:57:13.954484 20444 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:13.954567 20444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:13.954738 20444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:13.955950 20444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/data/instance:
uuid: "65eb0cc40157443f86c7d1e0b3bfc688"
format_stamp: "Formatted at 2026-08-10 14:57:13 on dist-test-slave-0b58"
server_key: "7cde1a59f81c675141754d32a7effb2d"
server_key_iv: "b3826688e438e49e1dd23b36503da012"
server_key_version: "encryptionkey@0"
I20260810 14:57:13.956444 20444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance:
uuid: "65eb0cc40157443f86c7d1e0b3bfc688"
format_stamp: "Formatted at 2026-08-10 14:57:13 on dist-test-slave-0b58"
server_key: "7cde1a59f81c675141754d32a7effb2d"
server_key_iv: "b3826688e438e49e1dd23b36503da012"
server_key_version: "encryptionkey@0"
I20260810 14:57:13.957969 20444 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:57:13.958839 20460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:13.959056 20444 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:57:13.959158 20444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/wal
uuid: "65eb0cc40157443f86c7d1e0b3bfc688"
format_stamp: "Formatted at 2026-08-10 14:57:13 on dist-test-slave-0b58"
server_key: "7cde1a59f81c675141754d32a7effb2d"
server_key_iv: "b3826688e438e49e1dd23b36503da012"
server_key_version: "encryptionkey@0"
I20260810 14:57:13.959239 20444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:13.975039 20444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:13.975379 20444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:13.975497 20444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:13.975716 20444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:57:13.976117 20444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:57:13.976156 20444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:13.976181 20444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:57:13.976240 20444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:13.982586 20444 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.67:44561
I20260810 14:57:13.982692 20585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.67:44561 every 8 connection(s)
I20260810 14:57:13.983188 20444 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/data/info.pb
I20260810 14:57:13.988334 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 20444
I20260810 14:57:13.988487 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/wal/instance
I20260810 14:57:13.988699 31017 external_mini_cluster.cc:1428] Setting key 56f43073d2364d7b6b5f67188dc5d107
I20260810 14:57:13.989938 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.984157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.67:38253 (local address 127.30.74.126:39827)
0810 14:57:13.984345 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:13.984348 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:13.984631 (+ 283us) server_negotiation.cc:407] Connection header received
0810 14:57:13.985249 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:13.985250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:13.985264 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:13.985356 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:13.986236 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.986968 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.988005 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.988304 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.988431 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.989582 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:13.989583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:13.989584 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:13.989589 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:13.989609 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:13.989614 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:13.989690 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:13.989796 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 14:57:13.989836 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 14:57:13.990142 20588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:13.984044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:39827 (local address 127.30.74.67:38253)
0810 14:57:13.984474 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:13.984513 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:13.985048 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:13.985499 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:13.985504 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:13.985513 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:13.986088 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.986103 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.987064 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.987065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.987911 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:13.987913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:13.988482 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:13.988484 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:13.988665 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:13.989457 (+ 792us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:57:13.989457 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:13.989489 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:57:13.989515 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:13.989689 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:13.989691 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:13.989694 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:57:13.989782 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 14:57:13.989910 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":123,"threads_started":1}
I20260810 14:57:13.990607 20586 heartbeater.cc:347] Connected to a master server at 127.30.74.126:39827
I20260810 14:57:13.990710 20586 heartbeater.cc:464] Registering TS with master...
I20260810 14:57:13.990875 20586 heartbeater.cc:511] Master 127.30.74.126:39827 requested a full tablet report, sending...
I20260810 14:57:13.991283 20075 ts_manager.cc:194] Registered new tserver with Master: 65eb0cc40157443f86c7d1e0b3bfc688 (127.30.74.67:44561)
I20260810 14:57:13.991716 20075 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.74.67:38253
I20260810 14:57:14.001571 31017 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:57:14.006623 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:14.003287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41548 (local address 127.30.74.126:39827)
0810 14:57:14.003387 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:14.003390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:14.003497 (+ 107us) server_negotiation.cc:407] Connection header received
0810 14:57:14.003584 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:14.003585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:14.003605 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:14.003702 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:14.003898 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.004482 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.005274 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.005468 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.005546 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:14.006321 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:14.006322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:14.006323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:14.006328 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:14.006343 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:14.006347 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:14.006411 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:14.006476 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 14:57:14.006512 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 14:57:14.008878 20075 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:57:14.010703 20075 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:57:14.018375 20597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:14.013870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45754 (local address 127.30.74.65:37089)
0810 14:57:14.014464 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:14.014468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:14.014477 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:57:14.014515 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:14.014517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:14.014531 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:14.014775 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:14.014843 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.015576 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.017079 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.017857 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.017941 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:14.018049 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:14.018222 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 14:57:14.018264 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":94,"threads_started":1}
I20260810 14:57:14.018348 20596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:14.013847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.66:35375 (local address 127.0.0.1:35958)
0810 14:57:14.014295 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:14.014304 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:14.014344 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:14.014641 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:14.014642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:14.014643 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:14.014693 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:14.014696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.016147 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.016148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:14.017270 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:14.017273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.018114 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.018115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:14.018175 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:14.018179 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:57:14.018226 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 14:57:14.018256 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":84,"thread_start_us":205,"threads_started":1}
I20260810 14:57:14.018357 20595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:14.013768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:37089 (local address 127.0.0.1:45754)
0810 14:57:14.013992 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:14.014038 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:14.014191 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:14.014625 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:14.014626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:14.014629 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:14.014694 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:14.014696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.015681 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.015683 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:14.016964 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:14.016968 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.017964 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.017966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:14.018057 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:14.018071 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 14:57:14.018161 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 14:57:14.018235 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":4,"thread_start_us":88,"threads_started":1}
I20260810 14:57:14.018697 20599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:14.014100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58142 (local address 127.30.74.67:44561)
0810 14:57:14.014361 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:14.014366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:14.014379 (+ 13us) server_negotiation.cc:407] Connection header received
0810 14:57:14.014425 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:14.014427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:14.014445 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:14.014760 (+ 315us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:14.014846 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.016055 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.017352 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.018106 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.018413 (+ 307us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:14.018519 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:14.018549 (+ 30us) server_negotiation.cc:299] Negotiation successful
0810 14:57:14.018600 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":96,"threads_started":1}
I20260810 14:57:14.018767 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:14.014201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35958 (local address 127.30.74.66:35375)
0810 14:57:14.014391 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:14.014396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:14.014408 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:57:14.014455 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:14.014458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:14.014474 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:14.014556 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:14.014771 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.016024 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.017361 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.017883 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.017982 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:14.018078 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:14.018633 (+ 555us) server_negotiation.cc:299] Negotiation successful
0810 14:57:14.018682 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
I20260810 14:57:14.019006 20598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:14.013819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:44561 (local address 127.0.0.1:58142)
0810 14:57:14.014113 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:14.014123 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:14.014191 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:14.014563 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:14.014567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:14.014570 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:14.014656 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:14.014658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.016162 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.016163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:14.017240 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:14.017243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.018218 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.018219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:14.018282 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:14.018287 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:57:14.018320 (+ 33us) client_negotiation.cc:240] Negotiation successful
0810 14:57:14.018362 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":141,"threads_started":1}
I20260810 14:57:14.019629 20365 tablet_service.cc:1467] Processing CreateTablet for tablet 86e9bfb9b38d404c8c843538fa2b46e8 (DEFAULT_TABLE table=test-table [id=cdcf2af4c465407599574d0cf06583a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:14.019629 20214 tablet_service.cc:1467] Processing CreateTablet for tablet 86e9bfb9b38d404c8c843538fa2b46e8 (DEFAULT_TABLE table=test-table [id=cdcf2af4c465407599574d0cf06583a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:14.019704 20512 tablet_service.cc:1467] Processing CreateTablet for tablet 86e9bfb9b38d404c8c843538fa2b46e8 (DEFAULT_TABLE table=test-table [id=cdcf2af4c465407599574d0cf06583a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:57:14.020061 20512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86e9bfb9b38d404c8c843538fa2b46e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:14.020054 20214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86e9bfb9b38d404c8c843538fa2b46e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:14.020054 20365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86e9bfb9b38d404c8c843538fa2b46e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:14.023140 20601 tablet_bootstrap.cc:492] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688: Bootstrap starting.
I20260810 14:57:14.023707 20603 tablet_bootstrap.cc:492] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b: Bootstrap starting.
I20260810 14:57:14.024055 20601 tablet_bootstrap.cc:654] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:14.024415 20602 tablet_bootstrap.cc:492] T 86e9bfb9b38d404c8c843538fa2b46e8 P c353cc3825dc47099dea090f908f063c: Bootstrap starting.
I20260810 14:57:14.024479 20603 tablet_bootstrap.cc:654] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:14.024458 20601 log.cc:824] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:14.024819 20603 log.cc:824] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:14.025285 20602 tablet_bootstrap.cc:654] T 86e9bfb9b38d404c8c843538fa2b46e8 P c353cc3825dc47099dea090f908f063c: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:14.025422 20601 tablet_bootstrap.cc:492] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688: No bootstrap required, opened a new log
I20260810 14:57:14.025508 20601 ts_tablet_manager.cc:1397] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260810 14:57:14.025630 20603 tablet_bootstrap.cc:492] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b: No bootstrap required, opened a new log
I20260810 14:57:14.025628 20602 log.cc:824] T 86e9bfb9b38d404c8c843538fa2b46e8 P c353cc3825dc47099dea090f908f063c: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:14.025724 20603 ts_tablet_manager.cc:1397] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:57:14.026546 20602 tablet_bootstrap.cc:492] T 86e9bfb9b38d404c8c843538fa2b46e8 P c353cc3825dc47099dea090f908f063c: No bootstrap required, opened a new log
I20260810 14:57:14.026628 20602 ts_tablet_manager.cc:1397] T 86e9bfb9b38d404c8c843538fa2b46e8 P c353cc3825dc47099dea090f908f063c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:57:14.028102 20603 raft_consensus.cc:348] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65eb0cc40157443f86c7d1e0b3bfc688" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 44561 } } peers { permanent_uuid: "c353cc3825dc47099dea090f908f063c" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37089 } } peers { permanent_uuid: "40c0a3915e224f0698a76912165d871b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35375 } }
I20260810 14:57:14.028298 20603 raft_consensus.cc:374] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:14.028347 20603 raft_consensus.cc:729] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c0a3915e224f0698a76912165d871b, State: Initialized, Role: FOLLOWER
I20260810 14:57:14.028514 20603 consensus_queue.cc:260] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65eb0cc40157443f86c7d1e0b3bfc688" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 44561 } } peers { permanent_uuid: "c353cc3825dc47099dea090f908f063c" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37089 } } peers { permanent_uuid: "40c0a3915e224f0698a76912165d871b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35375 } }
I20260810 14:57:14.028609 20601 raft_consensus.cc:348] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65eb0cc40157443f86c7d1e0b3bfc688" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 44561 } } peers { permanent_uuid: "c353cc3825dc47099dea090f908f063c" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37089 } } peers { permanent_uuid: "40c0a3915e224f0698a76912165d871b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35375 } }
I20260810 14:57:14.028788 20601 raft_consensus.cc:374] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:14.028836 20601 raft_consensus.cc:729] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65eb0cc40157443f86c7d1e0b3bfc688, State: Initialized, Role: FOLLOWER
I20260810 14:57:14.028843 20603 ts_tablet_manager.cc:1428] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:57:14.028978 20439 heartbeater.cc:503] Master 127.30.74.126:39827 was elected leader, sending a full tablet report...
I20260810 14:57:14.028959 20601 consensus_queue.cc:260] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65eb0cc40157443f86c7d1e0b3bfc688" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 44561 } } peers { permanent_uuid: "c353cc3825dc47099dea090f908f063c" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37089 } } peers { permanent_uuid: "40c0a3915e224f0698a76912165d871b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35375 } }
I20260810 14:57:14.029217 20601 ts_tablet_manager.cc:1428] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260810 14:57:14.029264 20586 heartbeater.cc:503] Master 127.30.74.126:39827 was elected leader, sending a full tablet report...
I20260810 14:57:14.029518 20602 raft_consensus.cc:348] T 86e9bfb9b38d404c8c843538fa2b46e8 P c353cc3825dc47099dea090f908f063c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65eb0cc40157443f86c7d1e0b3bfc688" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 44561 } } peers { permanent_uuid: "c353cc3825dc47099dea090f908f063c" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37089 } } peers { permanent_uuid: "40c0a3915e224f0698a76912165d871b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35375 } }
I20260810 14:57:14.029670 20602 raft_consensus.cc:374] T 86e9bfb9b38d404c8c843538fa2b46e8 P c353cc3825dc47099dea090f908f063c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:14.029704 20602 raft_consensus.cc:729] T 86e9bfb9b38d404c8c843538fa2b46e8 P c353cc3825dc47099dea090f908f063c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c353cc3825dc47099dea090f908f063c, State: Initialized, Role: FOLLOWER
I20260810 14:57:14.029871 20602 consensus_queue.cc:260] T 86e9bfb9b38d404c8c843538fa2b46e8 P c353cc3825dc47099dea090f908f063c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65eb0cc40157443f86c7d1e0b3bfc688" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 44561 } } peers { permanent_uuid: "c353cc3825dc47099dea090f908f063c" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37089 } } peers { permanent_uuid: "40c0a3915e224f0698a76912165d871b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35375 } }
I20260810 14:57:14.030237 20602 ts_tablet_manager.cc:1428] T 86e9bfb9b38d404c8c843538fa2b46e8 P c353cc3825dc47099dea090f908f063c: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 14:57:14.030287 20293 heartbeater.cc:503] Master 127.30.74.126:39827 was elected leader, sending a full tablet report...
W20260810 14:57:14.091295 20440 tablet.cc:2369] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:57:14.145177 20607 raft_consensus.cc:482] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:57:14.145354 20607 raft_consensus.cc:504] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65eb0cc40157443f86c7d1e0b3bfc688" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 44561 } } peers { permanent_uuid: "c353cc3825dc47099dea090f908f063c" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37089 } } peers { permanent_uuid: "40c0a3915e224f0698a76912165d871b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35375 } }
I20260810 14:57:14.145795 20607 leader_election.cc:288] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65eb0cc40157443f86c7d1e0b3bfc688 (127.30.74.67:44561), c353cc3825dc47099dea090f908f063c (127.30.74.65:37089)
I20260810 14:57:14.149083 20599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:14.146125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:53783 (local address 127.30.74.67:44561)
0810 14:57:14.146179 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:14.146181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:14.146190 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:57:14.146226 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:14.146226 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:14.146237 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:14.146300 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:14.146736 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.147234 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.148241 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.148683 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.148746 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:14.148782 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:14.148939 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 14:57:14.148993 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 14:57:14.149083 20443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:14.146000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.67:44561 (local address 127.30.74.66:53783)
0810 14:57:14.146048 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:14.146061 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:14.146133 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:14.146332 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:14.146333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:14.146335 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:14.146420 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:14.146422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.147357 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.147358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:14.148132 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:14.148134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.148779 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.148780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:14.148863 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:14.148867 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:57:14.148913 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 14:57:14.148968 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260810 14:57:14.149114 20597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:14.146185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.74.66:39521 (local address 127.30.74.65:37089)
0810 14:57:14.146254 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:14.146256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:14.146311 (+ 55us) server_negotiation.cc:407] Connection header received
0810 14:57:14.146371 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:14.146372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:14.146382 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:14.146452 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:57:14.146741 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.147233 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.148324 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.148696 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.148792 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:14.148801 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:14.148996 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 14:57:14.149048 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 14:57:14.149104 20611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:14.146036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:37089 (local address 127.30.74.66:39521)
0810 14:57:14.146257 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:14.146266 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:14.146315 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:14.146512 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:14.146513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:14.146516 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:57:14.146619 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:14.146622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.147349 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.147350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:14.148160 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:14.148162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.148811 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.148812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:14.148889 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:14.148892 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 14:57:14.148970 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 14:57:14.149007 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":2,"thread_start_us":128,"threads_started":1}
I20260810 14:57:14.149549 20239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86e9bfb9b38d404c8c843538fa2b46e8" candidate_uuid: "40c0a3915e224f0698a76912165d871b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c353cc3825dc47099dea090f908f063c" is_pre_election: true
I20260810 14:57:14.149552 20535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86e9bfb9b38d404c8c843538fa2b46e8" candidate_uuid: "40c0a3915e224f0698a76912165d871b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65eb0cc40157443f86c7d1e0b3bfc688" is_pre_election: true
I20260810 14:57:14.149730 20239 raft_consensus.cc:2454] T 86e9bfb9b38d404c8c843538fa2b46e8 P c353cc3825dc47099dea090f908f063c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40c0a3915e224f0698a76912165d871b in term 0.
I20260810 14:57:14.149726 20535 raft_consensus.cc:2454] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40c0a3915e224f0698a76912165d871b in term 0.
I20260810 14:57:14.149986 20316 leader_election.cc:302] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40c0a3915e224f0698a76912165d871b, c353cc3825dc47099dea090f908f063c; no voters:
I20260810 14:57:14.150121 20607 raft_consensus.cc:2781] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:57:14.150180 20607 raft_consensus.cc:482] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:57:14.150210 20607 raft_consensus.cc:3037] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:14.150954 20607 raft_consensus.cc:504] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65eb0cc40157443f86c7d1e0b3bfc688" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 44561 } } peers { permanent_uuid: "c353cc3825dc47099dea090f908f063c" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37089 } } peers { permanent_uuid: "40c0a3915e224f0698a76912165d871b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35375 } }
I20260810 14:57:14.151116 20607 leader_election.cc:288] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [CANDIDATE]: Term 1 election: Requested vote from peers 65eb0cc40157443f86c7d1e0b3bfc688 (127.30.74.67:44561), c353cc3825dc47099dea090f908f063c (127.30.74.65:37089)
I20260810 14:57:14.151324 20239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86e9bfb9b38d404c8c843538fa2b46e8" candidate_uuid: "40c0a3915e224f0698a76912165d871b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c353cc3825dc47099dea090f908f063c"
I20260810 14:57:14.151304 20535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86e9bfb9b38d404c8c843538fa2b46e8" candidate_uuid: "40c0a3915e224f0698a76912165d871b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65eb0cc40157443f86c7d1e0b3bfc688"
I20260810 14:57:14.151403 20535 raft_consensus.cc:3037] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:14.151429 20239 raft_consensus.cc:3037] T 86e9bfb9b38d404c8c843538fa2b46e8 P c353cc3825dc47099dea090f908f063c [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:14.152205 20239 raft_consensus.cc:2454] T 86e9bfb9b38d404c8c843538fa2b46e8 P c353cc3825dc47099dea090f908f063c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40c0a3915e224f0698a76912165d871b in term 1.
I20260810 14:57:14.152356 20535 raft_consensus.cc:2454] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40c0a3915e224f0698a76912165d871b in term 1.
I20260810 14:57:14.152456 20316 leader_election.cc:302] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40c0a3915e224f0698a76912165d871b, c353cc3825dc47099dea090f908f063c; no voters:
I20260810 14:57:14.152572 20607 raft_consensus.cc:2781] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:14.152731 20607 raft_consensus.cc:686] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [term 1 LEADER]: Becoming Leader. State: Replica: 40c0a3915e224f0698a76912165d871b, State: Running, Role: LEADER
I20260810 14:57:14.152843 20607 consensus_queue.cc:237] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65eb0cc40157443f86c7d1e0b3bfc688" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 44561 } } peers { permanent_uuid: "c353cc3825dc47099dea090f908f063c" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37089 } } peers { permanent_uuid: "40c0a3915e224f0698a76912165d871b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35375 } }
I20260810 14:57:14.153765 20073 catalog_manager.cc:5495] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b reported cstate change: term changed from 0 to 1, leader changed from <none> to 40c0a3915e224f0698a76912165d871b (127.30.74.66). New cstate: current_term: 1 leader_uuid: "40c0a3915e224f0698a76912165d871b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65eb0cc40157443f86c7d1e0b3bfc688" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 44561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c353cc3825dc47099dea090f908f063c" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40c0a3915e224f0698a76912165d871b" member_type: VOTER last_known_addr { host: "127.30.74.66" port: 35375 } health_report { overall_health: HEALTHY } } }
I20260810 14:57:14.168622 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:14.165982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35968 (local address 127.30.74.66:35375)
0810 14:57:14.166061 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:14.166064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:14.166074 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:57:14.166162 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:14.166163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:14.166182 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:14.166258 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:14.166424 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.166911 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.167707 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:14.167906 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:14.167982 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:14.168175 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:14.168332 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:14.168423 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:14.168468 (+ 45us) server_negotiation.cc:299] Negotiation successful
0810 14:57:14.168488 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 14:57:14.170804 20239 raft_consensus.cc:1261] T 86e9bfb9b38d404c8c843538fa2b46e8 P c353cc3825dc47099dea090f908f063c [term 1 FOLLOWER]: Refusing update from remote peer 40c0a3915e224f0698a76912165d871b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:57:14.170804 20535 raft_consensus.cc:1261] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688 [term 1 FOLLOWER]: Refusing update from remote peer 40c0a3915e224f0698a76912165d871b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:57:14.171237 20612 consensus_queue.cc:1035] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [LEADER]: Connected to new peer: Peer: permanent_uuid: "65eb0cc40157443f86c7d1e0b3bfc688" member_type: VOTER last_known_addr { host: "127.30.74.67" port: 44561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:57:14.171412 20607 consensus_queue.cc:1035] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c353cc3825dc47099dea090f908f063c" member_type: VOTER last_known_addr { host: "127.30.74.65" port: 37089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:57:14.174916 20616 mvcc.cc:205] Tried to move back new op lower bound from 7316987224760291328 to 7316987224690667520. Current Snapshot: MvccSnapshot[applied={T|T < 7316987224760291328}]
I20260810 14:57:14.175005 20617 mvcc.cc:205] Tried to move back new op lower bound from 7316987224760291328 to 7316987224690667520. Current Snapshot: MvccSnapshot[applied={T|T < 7316987224760291328}]
I20260810 14:57:14.177661 20075 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:57:14.177819 20075 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:57:14.178869 20075 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1bf640882974123b2a0e8a2d818399a: Sending DeleteTablet for 3 replicas of tablet 86e9bfb9b38d404c8c843538fa2b46e8
I20260810 14:57:14.179327 20365 tablet_service.cc:1514] Processing DeleteTablet for tablet 86e9bfb9b38d404c8c843538fa2b46e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:14 UTC) from {username='slave'} at 127.0.0.1:35958
I20260810 14:57:14.179317 20214 tablet_service.cc:1514] Processing DeleteTablet for tablet 86e9bfb9b38d404c8c843538fa2b46e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:14 UTC) from {username='slave'} at 127.0.0.1:45754
I20260810 14:57:14.179337 20512 tablet_service.cc:1514] Processing DeleteTablet for tablet 86e9bfb9b38d404c8c843538fa2b46e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:57:14 UTC) from {username='slave'} at 127.0.0.1:58142
I20260810 14:57:14.179880 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 20144
I20260810 14:57:14.188418 20629 tablet_replica.cc:330] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b: stopping tablet replica
I20260810 14:57:14.188591 20629 raft_consensus.cc:2229] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:57:14.188726 20629 raft_consensus.cc:2258] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:57:14.188848 20627 tablet_replica.cc:330] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688: stopping tablet replica
I20260810 14:57:14.189003 20627 raft_consensus.cc:2229] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:57:14.189148 20627 raft_consensus.cc:2258] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:57:14.189551 20629 ts_tablet_manager.cc:1905] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:57:14.189697 20627 ts_tablet_manager.cc:1905] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:57:14.191658 20629 ts_tablet_manager.cc:1918] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:57:14.191854 20629 log.cc:1196] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-1/wal/wals/86e9bfb9b38d404c8c843538fa2b46e8
I20260810 14:57:14.191881 20627 ts_tablet_manager.cc:1918] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:57:14.191968 20627 log.cc:1196] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688: Deleting WAL directory at /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373586526129-31017-0/minicluster-data/ts-2/wal/wals/86e9bfb9b38d404c8c843538fa2b46e8
I20260810 14:57:14.192297 20627 ts_tablet_manager.cc:1939] T 86e9bfb9b38d404c8c843538fa2b46e8 P 65eb0cc40157443f86c7d1e0b3bfc688: Deleting consensus metadata
I20260810 14:57:14.192448 20629 ts_tablet_manager.cc:1939] T 86e9bfb9b38d404c8c843538fa2b46e8 P 40c0a3915e224f0698a76912165d871b: Deleting consensus metadata
I20260810 14:57:14.192873 20060 catalog_manager.cc:4841] TS 65eb0cc40157443f86c7d1e0b3bfc688 (127.30.74.67:44561): tablet 86e9bfb9b38d404c8c843538fa2b46e8 (table test-table [id=cdcf2af4c465407599574d0cf06583a9]) successfully deleted
I20260810 14:57:14.192977 20058 catalog_manager.cc:4841] TS 40c0a3915e224f0698a76912165d871b (127.30.74.66:35375): tablet 86e9bfb9b38d404c8c843538fa2b46e8 (table test-table [id=cdcf2af4c465407599574d0cf06583a9]) successfully deleted
W20260810 14:57:14.193347 20059 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.74.65:37089 (error 108)
I20260810 14:57:14.193560 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 20297
I20260810 14:57:14.193890 20598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:14.193768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.65:37089 (local address 127.0.0.1:45766)
0810 14:57:14.193820 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:14.193857 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.65:37089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 14:57:14.196987 20059 catalog_manager.cc:4584] TS c353cc3825dc47099dea090f908f063c (127.30.74.65:37089): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 86e9bfb9b38d404c8c843538fa2b46e8: Network error: Client connection negotiation failed: client connection to 127.30.74.65:37089: connect: Connection refused (error 111)
I20260810 14:57:14.205502 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 20444
I20260810 14:57:14.213111 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 20040
2026-08-10T14:57:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (852 ms)
[----------] 6 tests from EncryptionPolicyTest (9057 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:14 dist-test-slave-0b58 krb5kdc[20633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:14 dist-test-slave-0b58 krb5kdc[20633](info): set up 2 sockets
Aug 10 14:57:14 dist-test-slave-0b58 krb5kdc[20633](info): commencing operation
krb5kdc: starting...
W20260810 14:57:15.319641 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:15 dist-test-slave-0b58 krb5kdc[20633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:57:15.460377 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35801
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:15.555058 20646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:15.555325 20646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:15.555352 20646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:15.556763 20646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:15.556823 20646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:15.556838 20646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:15.556851 20646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:15.556864 20646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:15.558616 20646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35801
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:15.558951 20646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:15.559234 20646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:15.564774 20656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:15.564805 20652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:15.564776 20653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:15.564970 20646 server_base.cc:1034] running on GCE node
I20260810 14:57:15.565451 20646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:15.565864 20646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:15.567050 20646 hybrid_clock.cc:648] HybridClock initialized: now 1786373835567000 us; error 73 us; skew 500 ppm
Aug 10 14:57:15 dist-test-slave-0b58 krb5kdc[20633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373835, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:15.575429 20646 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:57:15.575995 20646 webserver.cc:460] Webserver started at http://127.0.0.1:36445/ using document root <none> and password file <none>
I20260810 14:57:15.576344 20646 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:15.576424 20646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:15.576606 20646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:15.577760 20646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "338422ee68ff4527839a3120e780b91d"
format_stamp: "Formatted at 2026-08-10 14:57:15 on dist-test-slave-0b58"
server_key: "171287e9c41a547a58592c3be0e112ef"
server_key_iv: "357f2c78f8728e14f6975db3646fbf6b"
server_key_version: "encryptionkey@0"
I20260810 14:57:15.578163 20646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "338422ee68ff4527839a3120e780b91d"
format_stamp: "Formatted at 2026-08-10 14:57:15 on dist-test-slave-0b58"
server_key: "171287e9c41a547a58592c3be0e112ef"
server_key_iv: "357f2c78f8728e14f6975db3646fbf6b"
server_key_version: "encryptionkey@0"
I20260810 14:57:15.579725 20646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:57:15.580698 20664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:15.580988 20646 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:15.581149 20646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "338422ee68ff4527839a3120e780b91d"
format_stamp: "Formatted at 2026-08-10 14:57:15 on dist-test-slave-0b58"
server_key: "171287e9c41a547a58592c3be0e112ef"
server_key_iv: "357f2c78f8728e14f6975db3646fbf6b"
server_key_version: "encryptionkey@0"
I20260810 14:57:15.581246 20646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:15.604592 20646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:15.607084 20646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:15.607316 20646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:15.611755 20646 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35801
I20260810 14:57:15.611792 20721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35801 every 8 connection(s)
I20260810 14:57:15.612192 20646 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:15.613190 20722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:15.615566 20722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d: Bootstrap starting.
I20260810 14:57:15.616206 20722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:15.616552 20722 log.cc:824] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:15.617395 20722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d: No bootstrap required, opened a new log
I20260810 14:57:15.617455 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 20646
I20260810 14:57:15.617614 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:15.617841 31017 external_mini_cluster.cc:1428] Setting key 3d38adc3ee307e5072730611cacb38c5
I20260810 14:57:15.619446 20722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "338422ee68ff4527839a3120e780b91d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35801 } }
I20260810 14:57:15.619632 20722 raft_consensus.cc:374] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:15.619678 20722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 338422ee68ff4527839a3120e780b91d, State: Initialized, Role: FOLLOWER
I20260810 14:57:15.619820 20722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "338422ee68ff4527839a3120e780b91d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35801 } }
I20260810 14:57:15.619912 20722 raft_consensus.cc:388] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:15.619961 20722 raft_consensus.cc:482] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:15.620014 20722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:15.621102 20722 raft_consensus.cc:504] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "338422ee68ff4527839a3120e780b91d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35801 } }
I20260810 14:57:15.621235 20722 leader_election.cc:302] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 338422ee68ff4527839a3120e780b91d; no voters:
I20260810 14:57:15.621449 20722 leader_election.cc:288] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:15.621480 20729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:15.621596 20729 raft_consensus.cc:686] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [term 1 LEADER]: Becoming Leader. State: Replica: 338422ee68ff4527839a3120e780b91d, State: Running, Role: LEADER
I20260810 14:57:15.621763 20722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:15.621683 20729 consensus_queue.cc:237] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "338422ee68ff4527839a3120e780b91d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35801 } }
I20260810 14:57:15.622478 20733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "338422ee68ff4527839a3120e780b91d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "338422ee68ff4527839a3120e780b91d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35801 } } }
I20260810 14:57:15.622575 20733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:15.622759 20731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 338422ee68ff4527839a3120e780b91d. Latest consensus state: current_term: 1 leader_uuid: "338422ee68ff4527839a3120e780b91d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "338422ee68ff4527839a3120e780b91d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35801 } } }
I20260810 14:57:15.622833 20731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:15.622937 20740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:15.623481 20740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:15.625145 20740 catalog_manager.cc:1347] Generated new cluster ID: a549c86548a94cd98cf19d05c7320d6d
I20260810 14:57:15.625198 20740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 14:57:15 dist-test-slave-0b58 krb5kdc[20633](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 14:57:15.631647 20727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:15.618431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34577 (local address 127.0.0.1:35801)
0810 14:57:15.618643 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:15.618651 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:15.618678 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:57:15.619443 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:15.619452 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:15.619527 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:15.619698 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:15.620812 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:15.622087 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:15.623303 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:15.623909 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:15.624022 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:15.626997 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:15.627002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:15.627002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:15.627016 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:15.629238 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:15.629714 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:15.629718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:15.629718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:15.629866 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:15.630110 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:15.630111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:15.630112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:15.630463 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:15.631202 (+ 739us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:15.631300 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 14:57:15.631420 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":115,"threads_started":1}
I20260810 14:57:15.632910 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:15.636577 20740 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:15.637101 20740 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:15.639201 20740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 338422ee68ff4527839a3120e780b91d: Generated new TSK 0
I20260810 14:57:15.639410 20740 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:15.647292 20727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:15.636357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41297 (local address 127.0.0.1:35801)
0810 14:57:15.636420 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:15.636422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:15.638994 (+ 2572us) server_negotiation.cc:407] Connection header received
0810 14:57:15.639089 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:15.639090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:15.639103 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:15.639174 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:15.639991 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:15.640636 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:15.641788 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:15.642016 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:15.642078 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:15.643755 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:15.643757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:15.643757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:15.643765 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:15.645952 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:15.646434 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:15.646435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:15.646436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:15.646492 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:15.646743 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:15.646744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:15.646744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:15.646861 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:15.646946 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:15.647107 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 14:57:15.647170 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 14:57:15.649848 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 20646
W20260810 14:57:15.657913 20631 connection.cc:535] client connection to 127.0.0.1:35801 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35801): Connection reset by peer (error 104)
2026-08-10T14:57:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1436 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:15 dist-test-slave-0b58 krb5kdc[20761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:15 dist-test-slave-0b58 krb5kdc[20761](info): set up 2 sockets
Aug 10 14:57:15 dist-test-slave-0b58 krb5kdc[20761](info): commencing operation
krb5kdc: starting...
W20260810 14:57:16.776368 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:16 dist-test-slave-0b58 krb5kdc[20761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:57:16.911738 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34607
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:17.000478 20776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:17.000778 20776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:17.000823 20776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:17.002120 20776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:17.002219 20776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:17.002249 20776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:17.002279 20776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:17.002324 20776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:17.004094 20776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34607
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:17.004467 20776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:17.004732 20776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:17.010208 20783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:17.010416 20776 server_base.cc:1034] running on GCE node
W20260810 14:57:17.010214 20785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:17.010218 20781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:17.010915 20776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:17.011276 20776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:17.012449 20776 hybrid_clock.cc:648] HybridClock initialized: now 1786373837012442 us; error 44 us; skew 500 ppm
Aug 10 14:57:17 dist-test-slave-0b58 krb5kdc[20761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373837, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:17.015331 20776 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:57:17.015942 20776 webserver.cc:460] Webserver started at http://127.0.0.1:38535/ using document root <none> and password file <none>
I20260810 14:57:17.016227 20776 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:17.016378 20776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:17.016558 20776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:17.017933 20776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "a113accfb44a48fc80084fbfb8c3159e"
format_stamp: "Formatted at 2026-08-10 14:57:17 on dist-test-slave-0b58"
server_key: "5261270f5cb5d32d81a0fbd9feb625ab"
server_key_iv: "2b0dabcc2a55c7ed940fbba721fc9025"
server_key_version: "encryptionkey@0"
I20260810 14:57:17.018501 20776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "a113accfb44a48fc80084fbfb8c3159e"
format_stamp: "Formatted at 2026-08-10 14:57:17 on dist-test-slave-0b58"
server_key: "5261270f5cb5d32d81a0fbd9feb625ab"
server_key_iv: "2b0dabcc2a55c7ed940fbba721fc9025"
server_key_version: "encryptionkey@0"
I20260810 14:57:17.020852 20776 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260810 14:57:17.021917 20793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:17.022198 20776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:17.022324 20776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "a113accfb44a48fc80084fbfb8c3159e"
format_stamp: "Formatted at 2026-08-10 14:57:17 on dist-test-slave-0b58"
server_key: "5261270f5cb5d32d81a0fbd9feb625ab"
server_key_iv: "2b0dabcc2a55c7ed940fbba721fc9025"
server_key_version: "encryptionkey@0"
I20260810 14:57:17.022439 20776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:17.033289 20776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:17.035527 20776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:17.035717 20776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:17.040436 20776 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34607
I20260810 14:57:17.040477 20851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34607 every 8 connection(s)
I20260810 14:57:17.040901 20776 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:17.041910 20852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:17.044580 20852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e: Bootstrap starting.
I20260810 14:57:17.045279 20852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:17.045612 20852 log.cc:824] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:17.046896 20852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e: No bootstrap required, opened a new log
I20260810 14:57:17.048584 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 20776
I20260810 14:57:17.048697 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:17.048928 31017 external_mini_cluster.cc:1428] Setting key 784b0d25769ff907ab8ad1f3d49c0f81
I20260810 14:57:17.049265 20852 raft_consensus.cc:348] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a113accfb44a48fc80084fbfb8c3159e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34607 } }
I20260810 14:57:17.049685 20852 raft_consensus.cc:374] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:17.049788 20852 raft_consensus.cc:729] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a113accfb44a48fc80084fbfb8c3159e, State: Initialized, Role: FOLLOWER
I20260810 14:57:17.050014 20852 consensus_queue.cc:260] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a113accfb44a48fc80084fbfb8c3159e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34607 } }
I20260810 14:57:17.050117 20852 raft_consensus.cc:388] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:17.050151 20852 raft_consensus.cc:482] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:17.050194 20852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:17.051601 20852 raft_consensus.cc:504] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a113accfb44a48fc80084fbfb8c3159e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34607 } }
I20260810 14:57:17.051891 20852 leader_election.cc:302] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a113accfb44a48fc80084fbfb8c3159e; no voters:
I20260810 14:57:17.052116 20852 leader_election.cc:288] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:17.052309 20857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:17.052424 20852 sys_catalog.cc:564] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:17.052491 20857 raft_consensus.cc:686] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [term 1 LEADER]: Becoming Leader. State: Replica: a113accfb44a48fc80084fbfb8c3159e, State: Running, Role: LEADER
I20260810 14:57:17.052647 20857 consensus_queue.cc:237] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a113accfb44a48fc80084fbfb8c3159e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34607 } }
I20260810 14:57:17.053144 20858 sys_catalog.cc:455] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a113accfb44a48fc80084fbfb8c3159e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a113accfb44a48fc80084fbfb8c3159e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34607 } } }
I20260810 14:57:17.053164 20859 sys_catalog.cc:455] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a113accfb44a48fc80084fbfb8c3159e. Latest consensus state: current_term: 1 leader_uuid: "a113accfb44a48fc80084fbfb8c3159e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a113accfb44a48fc80084fbfb8c3159e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34607 } } }
I20260810 14:57:17.053386 20858 sys_catalog.cc:458] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:17.053458 20859 sys_catalog.cc:458] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e [sys.catalog]: This master's current role is: LEADER
W20260810 14:57:17.054780 20873 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:57:17.054884 20873 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:57:17.055222 20868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:17.055483 20868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:17.056288 20856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:17.049699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50615 (local address 127.0.0.1:34607)
0810 14:57:17.049925 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:17.049936 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:17.049973 (+ 37us) server_negotiation.cc:407] Connection header received
0810 14:57:17.050101 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:17.050110 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:17.050174 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:17.050344 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:17.050568 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:17.052212 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:17.053540 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:17.054249 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:17.054403 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:17.055423 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:17.055428 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:17.055429 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:17.055658 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:17.055700 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:17.055717 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:17.055812 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:17.055961 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 14:57:17.055995 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":116,"threads_started":1}
I20260810 14:57:17.057379 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:17.062523 20856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:17.058900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44807 (local address 127.0.0.1:34607)
0810 14:57:17.058945 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:17.058947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:17.059046 (+ 99us) server_negotiation.cc:407] Connection header received
0810 14:57:17.059132 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:17.059133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:17.059141 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:17.059193 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:17.059349 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:17.059838 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:17.061117 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:17.061355 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:17.061422 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:17.062202 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:17.062203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:17.062203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:17.062208 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:17.062224 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:17.062228 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:17.062280 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:17.062390 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 14:57:17.062404 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 14:57:17.063283 20868 catalog_manager.cc:1347] Generated new cluster ID: 290440df762c484eb81b6a0bcc207f19
I20260810 14:57:17.063354 20868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:17.087857 20868 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:17.088560 20868 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:17.095424 20868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a113accfb44a48fc80084fbfb8c3159e: Generated new TSK 0
I20260810 14:57:17.095679 20868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:17.102316 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 20776
W20260810 14:57:17.111687 20759 connection.cc:535] client connection to 127.0.0.1:34607 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34607): Connection reset by peer (error 104)
2026-08-10T14:57:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1454 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:17 dist-test-slave-0b58 krb5kdc[20885](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:17 dist-test-slave-0b58 krb5kdc[20885](info): set up 2 sockets
Aug 10 14:57:17 dist-test-slave-0b58 krb5kdc[20885](info): commencing operation
krb5kdc: starting...
W20260810 14:57:18.213210 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:18 dist-test-slave-0b58 krb5kdc[20885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:57:18.363992 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46073
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:18.453040 20902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:18.453435 20902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:18.453553 20902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:18.454978 20902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:18.455065 20902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:18.455090 20902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:18.455113 20902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:18.455149 20902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:18.456979 20902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46073
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:18.457330 20902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:18.457706 20902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:18.463353 20909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:18.463370 20912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:18.463434 20910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:18.463703 20902 server_base.cc:1034] running on GCE node
I20260810 14:57:18.464020 20902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:18.464430 20902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:18.465613 20902 hybrid_clock.cc:648] HybridClock initialized: now 1786373838465600 us; error 33 us; skew 500 ppm
Aug 10 14:57:18 dist-test-slave-0b58 krb5kdc[20885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:18.468550 20902 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:57:18.469094 20902 webserver.cc:460] Webserver started at http://127.0.0.1:40027/ using document root <none> and password file <none>
I20260810 14:57:18.469380 20902 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:18.469456 20902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:18.469615 20902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:18.470916 20902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "63c973d002504fcf82d7586f4affe10a"
format_stamp: "Formatted at 2026-08-10 14:57:18 on dist-test-slave-0b58"
server_key: "8ae3641befc199403e3acc99cd9dceb1"
server_key_iv: "d1236e92735b143a6880d2be21a7816b"
server_key_version: "encryptionkey@0"
I20260810 14:57:18.471355 20902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "63c973d002504fcf82d7586f4affe10a"
format_stamp: "Formatted at 2026-08-10 14:57:18 on dist-test-slave-0b58"
server_key: "8ae3641befc199403e3acc99cd9dceb1"
server_key_iv: "d1236e92735b143a6880d2be21a7816b"
server_key_version: "encryptionkey@0"
I20260810 14:57:18.472895 20902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:57:18.473773 20919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:18.473985 20902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 14:57:18.474102 20902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "63c973d002504fcf82d7586f4affe10a"
format_stamp: "Formatted at 2026-08-10 14:57:18 on dist-test-slave-0b58"
server_key: "8ae3641befc199403e3acc99cd9dceb1"
server_key_iv: "d1236e92735b143a6880d2be21a7816b"
server_key_version: "encryptionkey@0"
I20260810 14:57:18.474180 20902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:18.477743 20902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:18.480348 20902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:18.480523 20902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:18.484642 20902 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46073
I20260810 14:57:18.484731 20977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46073 every 8 connection(s)
I20260810 14:57:18.485067 20902 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:18.486145 20978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:18.488744 20978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a: Bootstrap starting.
I20260810 14:57:18.489398 20978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:18.489711 20978 log.cc:824] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:18.490460 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 20902
I20260810 14:57:18.490595 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:18.490657 20978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a: No bootstrap required, opened a new log
I20260810 14:57:18.490795 31017 external_mini_cluster.cc:1428] Setting key a0c94e31c5ebb36a1410e6b3e7b7e49b
I20260810 14:57:18.493348 20978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c973d002504fcf82d7586f4affe10a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46073 } }
I20260810 14:57:18.493542 20978 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:18.493569 20978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63c973d002504fcf82d7586f4affe10a, State: Initialized, Role: FOLLOWER
I20260810 14:57:18.493726 20978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c973d002504fcf82d7586f4affe10a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46073 } }
I20260810 14:57:18.493813 20978 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:18.493836 20978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:18.493909 20978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:18.493882 20982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:18.491410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39217 (local address 127.0.0.1:46073)
0810 14:57:18.491621 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:18.491629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:18.491655 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:57:18.492187 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:18.492195 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:18.492227 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:18.492419 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:18.493131 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:18.493134 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:18.493134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:18.493138 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:18.493178 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:18.493189 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:18.493254 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:18.493349 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 14:57:18.493414 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":102,"threads_started":1}
I20260810 14:57:18.494850 20978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c973d002504fcf82d7586f4affe10a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46073 } }
I20260810 14:57:18.494977 20978 leader_election.cc:302] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63c973d002504fcf82d7586f4affe10a; no voters:
I20260810 14:57:18.494989 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:18.495163 20978 leader_election.cc:288] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:18.495211 20985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:18.495396 20985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [term 1 LEADER]: Becoming Leader. State: Replica: 63c973d002504fcf82d7586f4affe10a, State: Running, Role: LEADER
I20260810 14:57:18.495494 20978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:18.495548 20985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c973d002504fcf82d7586f4affe10a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46073 } }
I20260810 14:57:18.495895 20985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63c973d002504fcf82d7586f4affe10a. Latest consensus state: current_term: 1 leader_uuid: "63c973d002504fcf82d7586f4affe10a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c973d002504fcf82d7586f4affe10a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46073 } } }
I20260810 14:57:18.495904 20987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63c973d002504fcf82d7586f4affe10a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c973d002504fcf82d7586f4affe10a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46073 } } }
I20260810 14:57:18.496057 20985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:18.496078 20987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:18.496492 21000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:18.497082 21000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:18.499024 21010 mvcc.cc:205] Tried to move back new op lower bound from 7316987242488999936 to 7316987242478202880. Current Snapshot: MvccSnapshot[applied={T|T < 7316987242488999936}]
I20260810 14:57:18.499246 21000 catalog_manager.cc:1347] Generated new cluster ID: c7bdac8149224058aabe8dd0e3e28992
I20260810 14:57:18.499310 21000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:18.499372 20982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:18.497421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40985 (local address 127.0.0.1:46073)
0810 14:57:18.497463 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:18.497466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:18.497811 (+ 345us) server_negotiation.cc:407] Connection header received
0810 14:57:18.497935 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:18.497936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:18.497943 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:18.498013 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:18.498978 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:18.498979 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:18.498980 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:18.498984 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:18.499011 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:18.499017 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:18.499081 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:18.499237 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 14:57:18.499254 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:57:18.511785 21000 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:18.511999 21000 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:18.516111 21000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63c973d002504fcf82d7586f4affe10a: Generated new TSK 0
I20260810 14:57:18.516386 21000 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:18.519196 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 20902
2026-08-10T14:57:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1415 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:18 dist-test-slave-0b58 krb5kdc[21014](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:18 dist-test-slave-0b58 krb5kdc[21014](info): set up 2 sockets
Aug 10 14:57:18 dist-test-slave-0b58 krb5kdc[21014](info): commencing operation
krb5kdc: starting...
W20260810 14:57:19.640074 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:19 dist-test-slave-0b58 krb5kdc[21014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:57:19.784093 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44589
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:19.870482 21029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:19.870781 21029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:19.870831 21029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:19.872193 21029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:19.872331 21029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:19.872388 21029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:19.872417 21029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:19.872447 21029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:19.874399 21029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:19.874742 21029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:19.875017 21029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:19.880506 21037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:19.880517 21035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:19.880517 21039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:19.880929 21029 server_base.cc:1034] running on GCE node
I20260810 14:57:19.881208 21029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:19.881559 21029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:19.882746 21029 hybrid_clock.cc:648] HybridClock initialized: now 1786373839882739 us; error 59 us; skew 500 ppm
Aug 10 14:57:19 dist-test-slave-0b58 krb5kdc[21014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373839, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:19.885568 21029 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:57:19.886107 21029 webserver.cc:460] Webserver started at http://127.0.0.1:34989/ using document root <none> and password file <none>
I20260810 14:57:19.886384 21029 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:19.886435 21029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:19.886570 21029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:19.887725 21029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "989ef3d897344265b9f86dff19aec86b"
format_stamp: "Formatted at 2026-08-10 14:57:19 on dist-test-slave-0b58"
server_key: "cc92f71ca76711a7d5550ab5c65ea292"
server_key_iv: "cfcfd815d03dbd4ccf48b545cd16d8c2"
server_key_version: "encryptionkey@0"
I20260810 14:57:19.888137 21029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "989ef3d897344265b9f86dff19aec86b"
format_stamp: "Formatted at 2026-08-10 14:57:19 on dist-test-slave-0b58"
server_key: "cc92f71ca76711a7d5550ab5c65ea292"
server_key_iv: "cfcfd815d03dbd4ccf48b545cd16d8c2"
server_key_version: "encryptionkey@0"
I20260810 14:57:19.889750 21029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:57:19.890750 21048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:19.891058 21029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:57:19.891186 21029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "989ef3d897344265b9f86dff19aec86b"
format_stamp: "Formatted at 2026-08-10 14:57:19 on dist-test-slave-0b58"
server_key: "cc92f71ca76711a7d5550ab5c65ea292"
server_key_iv: "cfcfd815d03dbd4ccf48b545cd16d8c2"
server_key_version: "encryptionkey@0"
I20260810 14:57:19.891273 21029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:19.908756 21029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:19.911413 21029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:19.911620 21029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:19.916410 21029 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44589
I20260810 14:57:19.916476 21103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44589 every 8 connection(s)
I20260810 14:57:19.916882 21029 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:19.917902 21104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:19.920528 21104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b: Bootstrap starting.
I20260810 14:57:19.920722 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 21029
I20260810 14:57:19.920866 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:19.921068 31017 external_mini_cluster.cc:1428] Setting key e6b8dd368d4d3b8dff7f209fec7488b8
I20260810 14:57:19.921391 21104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:19.921703 21104 log.cc:824] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:19.922566 21104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b: No bootstrap required, opened a new log
I20260810 14:57:19.924633 21104 raft_consensus.cc:348] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989ef3d897344265b9f86dff19aec86b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44589 } }
I20260810 14:57:19.924818 21104 raft_consensus.cc:374] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:19.924872 21104 raft_consensus.cc:729] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 989ef3d897344265b9f86dff19aec86b, State: Initialized, Role: FOLLOWER
I20260810 14:57:19.925025 21104 consensus_queue.cc:260] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989ef3d897344265b9f86dff19aec86b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44589 } }
I20260810 14:57:19.925115 21104 raft_consensus.cc:388] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:19.925163 21104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:19.925215 21104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:19.925863 21104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989ef3d897344265b9f86dff19aec86b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44589 } }
I20260810 14:57:19.925992 21104 leader_election.cc:302] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 989ef3d897344265b9f86dff19aec86b; no voters:
I20260810 14:57:19.926239 21104 leader_election.cc:288] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:19.926275 21110 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:19.926455 21110 raft_consensus.cc:686] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [term 1 LEADER]: Becoming Leader. State: Replica: 989ef3d897344265b9f86dff19aec86b, State: Running, Role: LEADER
I20260810 14:57:19.926544 21110 consensus_queue.cc:237] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989ef3d897344265b9f86dff19aec86b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44589 } }
I20260810 14:57:19.926599 21104 sys_catalog.cc:564] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:19.927309 21110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 989ef3d897344265b9f86dff19aec86b. Latest consensus state: current_term: 1 leader_uuid: "989ef3d897344265b9f86dff19aec86b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989ef3d897344265b9f86dff19aec86b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44589 } } }
I20260810 14:57:19.927320 21112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "989ef3d897344265b9f86dff19aec86b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "989ef3d897344265b9f86dff19aec86b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44589 } } }
I20260810 14:57:19.927415 21110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:19.927438 21112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b [sys.catalog]: This master's current role is: LEADER
W20260810 14:57:19.928784 21127 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:57:19.928892 21127 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:57:19.928972 21119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:19.929190 21119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 14:57:19 dist-test-slave-0b58 krb5kdc[21014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 14:57:19.931046 21119 catalog_manager.cc:1347] Generated new cluster ID: c1458bf48dcf4ce8b066a295c2f0d8b0
I20260810 14:57:19.931113 21119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:19.934443 21108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:19.921689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47551 (local address 127.0.0.1:44589)
0810 14:57:19.921906 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:19.921918 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:19.921967 (+ 49us) server_negotiation.cc:407] Connection header received
0810 14:57:19.922568 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:19.922576 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:19.922665 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:19.922840 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:19.923558 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:19.924553 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:19.925679 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:19.926290 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:19.926835 (+ 545us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:19.929832 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:19.929837 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:19.929837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:19.929852 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:19.932176 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:19.932672 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:19.932677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:19.932677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:19.932788 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:19.933001 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:19.933002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:19.933003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:19.933351 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:19.933526 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:19.934036 (+ 510us) server_negotiation.cc:299] Negotiation successful
0810 14:57:19.934203 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
I20260810 14:57:19.941043 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:19.941104 21119 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:19.941910 21119 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:19.953516 21119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 989ef3d897344265b9f86dff19aec86b: Generated new TSK 0
I20260810 14:57:19.953775 21119 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:19.956135 21108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:19.942696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42327 (local address 127.0.0.1:44589)
0810 14:57:19.942762 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:19.942764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:19.944501 (+ 1737us) server_negotiation.cc:407] Connection header received
0810 14:57:19.944587 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:19.944588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:19.944601 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:19.944655 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:19.945288 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:19.945994 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:19.947239 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:19.947623 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:19.947671 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:19.949649 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:19.949651 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:19.949651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:19.949660 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:19.954409 (+ 4749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:19.954969 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:19.954971 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:19.954971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:19.955044 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:19.955284 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:19.955285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:19.955285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:19.955434 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:19.955498 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:19.955898 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 14:57:19.956010 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 14:57:19.958727 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 21029
2026-08-10T14:57:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1437 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:20 dist-test-slave-0b58 krb5kdc[21142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:20 dist-test-slave-0b58 krb5kdc[21142](info): set up 2 sockets
Aug 10 14:57:20 dist-test-slave-0b58 krb5kdc[21142](info): commencing operation
krb5kdc: starting...
W20260810 14:57:21.068358 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:21 dist-test-slave-0b58 krb5kdc[21142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:57:21.208376 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44809
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:21.297186 21157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:21.297438 21157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:21.297477 21157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:21.299023 21157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:21.299111 21157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:21.299127 21157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:21.299141 21157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:21.299155 21157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:21.301162 21157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:21.301504 21157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:21.301916 21157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:21.307492 21162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:21.307492 21165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:21.307493 21163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:21.307845 21157 server_base.cc:1034] running on GCE node
I20260810 14:57:21.308125 21157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:21.308525 21157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:21.309711 21157 hybrid_clock.cc:648] HybridClock initialized: now 1786373841309695 us; error 32 us; skew 500 ppm
Aug 10 14:57:21 dist-test-slave-0b58 krb5kdc[21142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373841, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:21.318622 21157 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:57:21.319147 21157 webserver.cc:460] Webserver started at http://127.0.0.1:35093/ using document root <none> and password file <none>
I20260810 14:57:21.319383 21157 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:21.319432 21157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:21.319536 21157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:21.320848 21157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "7238cb00cdf047ce8eaf8d6892356aef"
format_stamp: "Formatted at 2026-08-10 14:57:21 on dist-test-slave-0b58"
server_key: "e3cd75a0aca82aa5701036f943c2942c"
server_key_iv: "f3a685f6156dfac63a9a3a5a1c087461"
server_key_version: "encryptionkey@0"
I20260810 14:57:21.321205 21157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "7238cb00cdf047ce8eaf8d6892356aef"
format_stamp: "Formatted at 2026-08-10 14:57:21 on dist-test-slave-0b58"
server_key: "e3cd75a0aca82aa5701036f943c2942c"
server_key_iv: "f3a685f6156dfac63a9a3a5a1c087461"
server_key_version: "encryptionkey@0"
I20260810 14:57:21.322690 21157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:57:21.323536 21172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:21.323805 21157 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:21.323907 21157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "7238cb00cdf047ce8eaf8d6892356aef"
format_stamp: "Formatted at 2026-08-10 14:57:21 on dist-test-slave-0b58"
server_key: "e3cd75a0aca82aa5701036f943c2942c"
server_key_iv: "f3a685f6156dfac63a9a3a5a1c087461"
server_key_version: "encryptionkey@0"
I20260810 14:57:21.323982 21157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:21.356359 21157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:21.359097 21157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:21.359241 21157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:21.363474 21157 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44809
I20260810 14:57:21.363535 21230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44809 every 8 connection(s)
I20260810 14:57:21.363920 21157 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:21.364995 21231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:21.365477 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 21157
I20260810 14:57:21.365628 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:21.365882 31017 external_mini_cluster.cc:1428] Setting key c9e75f8a8682008f5a3a1cd369e8be06
I20260810 14:57:21.368145 21231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef: Bootstrap starting.
I20260810 14:57:21.368860 21231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:21.369168 21231 log.cc:824] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:21.369922 21231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef: No bootstrap required, opened a new log
I20260810 14:57:21.371891 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:21.366616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58929 (local address 127.0.0.1:44809)
0810 14:57:21.366855 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:21.366866 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:21.366908 (+ 42us) server_negotiation.cc:407] Connection header received
0810 14:57:21.367350 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:21.367357 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:21.367410 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:21.367575 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:21.367817 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:21.368830 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:21.369735 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:21.370243 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:21.370376 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:21.371227 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:21.371231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:21.371231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:21.371409 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:21.371442 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:21.371454 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:21.371532 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:21.371612 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 14:57:21.371693 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":144,"threads_started":1}
I20260810 14:57:21.372225 21231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7238cb00cdf047ce8eaf8d6892356aef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44809 } }
I20260810 14:57:21.372458 21231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:21.372505 21231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7238cb00cdf047ce8eaf8d6892356aef, State: Initialized, Role: FOLLOWER
I20260810 14:57:21.372637 21231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7238cb00cdf047ce8eaf8d6892356aef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44809 } }
I20260810 14:57:21.372735 21231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:21.372783 21231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:21.372834 21231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:21.372969 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:21.373557 21231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7238cb00cdf047ce8eaf8d6892356aef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44809 } }
I20260810 14:57:21.373656 21231 leader_election.cc:302] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7238cb00cdf047ce8eaf8d6892356aef; no voters:
I20260810 14:57:21.373880 21231 leader_election.cc:288] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:21.373914 21236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:21.374042 21236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [term 1 LEADER]: Becoming Leader. State: Replica: 7238cb00cdf047ce8eaf8d6892356aef, State: Running, Role: LEADER
I20260810 14:57:21.374135 21236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7238cb00cdf047ce8eaf8d6892356aef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44809 } }
I20260810 14:57:21.374169 21231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:21.374492 21236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7238cb00cdf047ce8eaf8d6892356aef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7238cb00cdf047ce8eaf8d6892356aef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44809 } } }
I20260810 14:57:21.374567 21236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:21.374689 21236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7238cb00cdf047ce8eaf8d6892356aef. Latest consensus state: current_term: 1 leader_uuid: "7238cb00cdf047ce8eaf8d6892356aef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7238cb00cdf047ce8eaf8d6892356aef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44809 } } }
I20260810 14:57:21.374768 21236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef [sys.catalog]: This master's current role is: LEADER
W20260810 14:57:21.376209 21260 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:57:21.376291 21260 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:57:21.376350 21249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:21.376518 21249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:21.378245 21249 catalog_manager.cc:1347] Generated new cluster ID: 7a029adf2f344224bf38f8a315dad5e1
I20260810 14:57:21.378314 21249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:21.378810 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:21.374677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54525 (local address 127.0.0.1:44809)
0810 14:57:21.374719 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:21.374721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:21.374943 (+ 222us) server_negotiation.cc:407] Connection header received
0810 14:57:21.375054 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:21.375056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:21.375068 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:21.375139 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:21.375301 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:21.375983 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:21.377063 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:21.377362 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:21.377435 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:21.378373 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:21.378374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:21.378375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:21.378381 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:21.378398 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:21.378403 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:21.378448 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:21.378698 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 14:57:21.378727 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 14:57:21.387146 21249 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:21.387745 21249 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:21.391808 21249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7238cb00cdf047ce8eaf8d6892356aef: Generated new TSK 0
I20260810 14:57:21.392069 21249 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:21.397506 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 21157
2026-08-10T14:57:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1440 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:21 dist-test-slave-0b58 krb5kdc[21266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:21 dist-test-slave-0b58 krb5kdc[21266](info): set up 2 sockets
Aug 10 14:57:21 dist-test-slave-0b58 krb5kdc[21266](info): commencing operation
krb5kdc: starting...
W20260810 14:57:22.513535 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:22 dist-test-slave-0b58 krb5kdc[21266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:57:22.661870 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41409
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:22.749603 21284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:22.749886 21284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:22.749929 21284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:22.751191 21284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:22.751274 21284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:22.751300 21284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:22.751323 21284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:22.751353 21284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:22.753306 21284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:22.753664 21284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:22.753933 21284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:22.759593 21289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:22.759766 21290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:22.759595 21292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:22.759877 21284 server_base.cc:1034] running on GCE node
I20260810 14:57:22.760165 21284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:22.760516 21284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:22.761693 21284 hybrid_clock.cc:648] HybridClock initialized: now 1786373842761656 us; error 49 us; skew 500 ppm
Aug 10 14:57:22 dist-test-slave-0b58 krb5kdc[21266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373842, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:22.764568 21284 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:57:22.765352 21284 webserver.cc:460] Webserver started at http://127.0.0.1:44237/ using document root <none> and password file <none>
I20260810 14:57:22.765674 21284 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:22.765741 21284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:22.765883 21284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:22.767071 21284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "dc559018c438414f93aa9e1132a26032"
format_stamp: "Formatted at 2026-08-10 14:57:22 on dist-test-slave-0b58"
server_key: "ddfc2fbc8ae25ef0ec1f5095b8322700"
server_key_iv: "129020f6c3b45fc8bb6b54e5abe10c64"
server_key_version: "encryptionkey@0"
I20260810 14:57:22.767530 21284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "dc559018c438414f93aa9e1132a26032"
format_stamp: "Formatted at 2026-08-10 14:57:22 on dist-test-slave-0b58"
server_key: "ddfc2fbc8ae25ef0ec1f5095b8322700"
server_key_iv: "129020f6c3b45fc8bb6b54e5abe10c64"
server_key_version: "encryptionkey@0"
I20260810 14:57:22.769229 21284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:57:22.770174 21299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:22.770543 21284 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:22.770689 21284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "dc559018c438414f93aa9e1132a26032"
format_stamp: "Formatted at 2026-08-10 14:57:22 on dist-test-slave-0b58"
server_key: "ddfc2fbc8ae25ef0ec1f5095b8322700"
server_key_iv: "129020f6c3b45fc8bb6b54e5abe10c64"
server_key_version: "encryptionkey@0"
I20260810 14:57:22.770761 21284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:22.774281 21284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:22.776738 21284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:22.776916 21284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:22.781013 21284 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41409
I20260810 14:57:22.781065 21355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41409 every 8 connection(s)
I20260810 14:57:22.781430 21284 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:22.782591 21357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:22.785154 21357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032: Bootstrap starting.
I20260810 14:57:22.785804 21357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:22.786114 21357 log.cc:824] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:22.786885 21357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032: No bootstrap required, opened a new log
I20260810 14:57:22.788520 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 21284
I20260810 14:57:22.788662 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:22.788870 31017 external_mini_cluster.cc:1428] Setting key f7d60596a0c874dac6357abf92180d2a
I20260810 14:57:22.788779 21357 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc559018c438414f93aa9e1132a26032" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41409 } }
I20260810 14:57:22.788972 21357 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:22.789018 21357 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc559018c438414f93aa9e1132a26032, State: Initialized, Role: FOLLOWER
I20260810 14:57:22.789172 21357 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc559018c438414f93aa9e1132a26032" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41409 } }
I20260810 14:57:22.789273 21357 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:22.789309 21357 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:22.789356 21357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:22.790548 21357 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc559018c438414f93aa9e1132a26032" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41409 } }
I20260810 14:57:22.790679 21357 leader_election.cc:302] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc559018c438414f93aa9e1132a26032; no voters:
I20260810 14:57:22.790863 21357 leader_election.cc:288] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:22.790925 21362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:22.791095 21362 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [term 1 LEADER]: Becoming Leader. State: Replica: dc559018c438414f93aa9e1132a26032, State: Running, Role: LEADER
I20260810 14:57:22.791155 21357 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:22.791208 21362 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc559018c438414f93aa9e1132a26032" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41409 } }
I20260810 14:57:22.791396 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:22.789447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33925 (local address 127.0.0.1:41409)
0810 14:57:22.789670 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:22.789688 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:22.789734 (+ 46us) server_negotiation.cc:407] Connection header received
0810 14:57:22.789894 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:22.789904 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:22.789939 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:22.790128 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:22.790866 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:22.790870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:22.790870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:22.790876 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:22.790927 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:22.790945 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:22.791029 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:22.791158 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 14:57:22.791205 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":104,"threads_started":1}
I20260810 14:57:22.791580 21362 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc559018c438414f93aa9e1132a26032. Latest consensus state: current_term: 1 leader_uuid: "dc559018c438414f93aa9e1132a26032" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc559018c438414f93aa9e1132a26032" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41409 } } }
I20260810 14:57:22.791682 21362 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:22.791589 21364 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc559018c438414f93aa9e1132a26032" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc559018c438414f93aa9e1132a26032" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41409 } } }
I20260810 14:57:22.791728 21364 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:22.792609 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:22.792740 21369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:22.793212 21369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:22.794641 21369 catalog_manager.cc:1347] Generated new cluster ID: 24e0e89127334a698cdf0c08eb312607
I20260810 14:57:22.794697 21369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:22.797930 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:22.796693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42319 (local address 127.0.0.1:41409)
0810 14:57:22.796730 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:22.796733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:22.796854 (+ 121us) server_negotiation.cc:407] Connection header received
0810 14:57:22.796921 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:22.796922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:22.796928 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:22.796999 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:22.797688 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:22.797689 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:22.797689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:22.797694 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:22.797710 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:22.797715 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:22.797771 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:22.797849 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 14:57:22.797863 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 14:57:22.808468 21369 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:22.808686 21369 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:22.813277 21369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc559018c438414f93aa9e1132a26032: Generated new TSK 0
I20260810 14:57:22.813565 21369 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:22.819846 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 21284
2026-08-10T14:57:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1421 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:22 dist-test-slave-0b58 krb5kdc[21390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:22 dist-test-slave-0b58 krb5kdc[21390](info): set up 2 sockets
Aug 10 14:57:22 dist-test-slave-0b58 krb5kdc[21390](info): commencing operation
krb5kdc: starting...
W20260810 14:57:23.945755 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:24 dist-test-slave-0b58 krb5kdc[21390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:57:24.092236 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46133
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:24.182454 21404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:24.182729 21404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:24.182756 21404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:24.184227 21404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:24.184330 21404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:24.184347 21404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:24.184361 21404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:24.184374 21404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:24.186671 21404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:24.187043 21404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:24.187359 21404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:24.192812 21415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:24.192833 21411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:24.192857 21404 server_base.cc:1034] running on GCE node
W20260810 14:57:24.192854 21413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:24.193300 21404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:24.193652 21404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:24.194911 21404 hybrid_clock.cc:648] HybridClock initialized: now 1786373844194900 us; error 34 us; skew 500 ppm
Aug 10 14:57:24 dist-test-slave-0b58 krb5kdc[21390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:24.198405 21404 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:57:24.198977 21404 webserver.cc:460] Webserver started at http://127.0.0.1:40425/ using document root <none> and password file <none>
I20260810 14:57:24.199247 21404 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:24.199299 21404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:24.199481 21404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:24.200760 21404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "f9d5803d6c224874931d591422cbbdb2"
format_stamp: "Formatted at 2026-08-10 14:57:24 on dist-test-slave-0b58"
server_key: "8f8b33d8b715e8067db1b69bb6b64d1f"
server_key_iv: "30c1c2132bddf873a69e979afc1e3d26"
server_key_version: "encryptionkey@0"
I20260810 14:57:24.201157 21404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "f9d5803d6c224874931d591422cbbdb2"
format_stamp: "Formatted at 2026-08-10 14:57:24 on dist-test-slave-0b58"
server_key: "8f8b33d8b715e8067db1b69bb6b64d1f"
server_key_iv: "30c1c2132bddf873a69e979afc1e3d26"
server_key_version: "encryptionkey@0"
I20260810 14:57:24.202766 21404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:57:24.203797 21423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:24.204041 21404 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:24.204175 21404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "f9d5803d6c224874931d591422cbbdb2"
format_stamp: "Formatted at 2026-08-10 14:57:24 on dist-test-slave-0b58"
server_key: "8f8b33d8b715e8067db1b69bb6b64d1f"
server_key_iv: "30c1c2132bddf873a69e979afc1e3d26"
server_key_version: "encryptionkey@0"
I20260810 14:57:24.204306 21404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:24.213132 21404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:24.215672 21404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:24.215822 21404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:24.220775 21404 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46133
I20260810 14:57:24.220813 21478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46133 every 8 connection(s)
I20260810 14:57:24.221264 21404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:24.222366 21479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:24.228240 21479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2: Bootstrap starting.
I20260810 14:57:24.229230 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 21404
I20260810 14:57:24.229415 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:24.229890 21479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:24.229951 31017 external_mini_cluster.cc:1428] Setting key a5a119f29d3fc22c579b9cb19c9c6735
I20260810 14:57:24.230614 21479 log.cc:824] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:24.232110 21479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2: No bootstrap required, opened a new log
I20260810 14:57:24.235360 21479 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9d5803d6c224874931d591422cbbdb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46133 } }
I20260810 14:57:24.235616 21479 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:24.235661 21479 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d5803d6c224874931d591422cbbdb2, State: Initialized, Role: FOLLOWER
I20260810 14:57:24.235828 21479 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9d5803d6c224874931d591422cbbdb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46133 } }
I20260810 14:57:24.235922 21479 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:24.235971 21479 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:24.236030 21479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:24.237025 21479 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9d5803d6c224874931d591422cbbdb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46133 } }
I20260810 14:57:24.237166 21479 leader_election.cc:302] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9d5803d6c224874931d591422cbbdb2; no voters:
I20260810 14:57:24.237401 21479 leader_election.cc:288] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:24.237762 21484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:24.237771 21479 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:24.238034 21484 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [term 1 LEADER]: Becoming Leader. State: Replica: f9d5803d6c224874931d591422cbbdb2, State: Running, Role: LEADER
I20260810 14:57:24.238135 21484 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9d5803d6c224874931d591422cbbdb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46133 } }
I20260810 14:57:24.238564 21485 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9d5803d6c224874931d591422cbbdb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9d5803d6c224874931d591422cbbdb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46133 } } }
I20260810 14:57:24.238672 21485 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:24.238883 21486 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9d5803d6c224874931d591422cbbdb2. Latest consensus state: current_term: 1 leader_uuid: "f9d5803d6c224874931d591422cbbdb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9d5803d6c224874931d591422cbbdb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46133 } } }
I20260810 14:57:24.238967 21486 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2 [sys.catalog]: This master's current role is: LEADER
Aug 10 14:57:24 dist-test-slave-0b58 krb5kdc[21390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
W20260810 14:57:24.240149 21502 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:57:24.240240 21502 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:57:24.240585 21495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:24.240821 21495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:24.242762 21495 catalog_manager.cc:1347] Generated new cluster ID: dbd37422efcb4171a40697bc2209fe64
I20260810 14:57:24.242827 21495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:24.245432 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:24.230835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:50979 (local address 127.0.0.1:46133)
0810 14:57:24.231090 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:24.231101 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:24.231143 (+ 42us) server_negotiation.cc:407] Connection header received
0810 14:57:24.231738 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:24.231753 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:24.231812 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:24.231938 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:24.233256 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:24.234200 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:24.235703 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:24.236453 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:24.236560 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:24.240119 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:24.240124 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:24.240124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:24.240139 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:24.242608 (+ 2469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:24.243202 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:24.243206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:24.243207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:24.243318 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:24.243563 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:24.243564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:24.243564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:24.244031 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:24.244243 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:24.244890 (+ 647us) server_negotiation.cc:299] Negotiation successful
0810 14:57:24.244996 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":116,"threads_started":1}
I20260810 14:57:24.246758 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:24.253710 21495 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:24.254483 21495 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:24.261406 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:24.248330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:32791 (local address 127.0.0.1:46133)
0810 14:57:24.248368 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:24.248370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:24.252517 (+ 4147us) server_negotiation.cc:407] Connection header received
0810 14:57:24.252655 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:24.252657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:24.252671 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:24.252754 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:24.253744 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:24.254532 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:24.255592 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:24.255962 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:24.256054 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:24.257695 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:24.257696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:24.257697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:24.257706 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:24.259650 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:24.260203 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:24.260204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:24.260205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:24.260442 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:24.260763 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:24.260764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:24.260765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:24.260929 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:24.261051 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:24.261213 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 14:57:24.261299 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 14:57:24.264729 21495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9d5803d6c224874931d591422cbbdb2: Generated new TSK 0
I20260810 14:57:24.264966 21495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:24.270777 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 21404
W20260810 14:57:24.277686 21388 connection.cc:535] client connection to 127.0.0.1:46133 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46133): Connection reset by peer (error 104)
2026-08-10T14:57:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1450 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:24 dist-test-slave-0b58 krb5kdc[21515](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:24 dist-test-slave-0b58 krb5kdc[21515](info): set up 2 sockets
Aug 10 14:57:24 dist-test-slave-0b58 krb5kdc[21515](info): commencing operation
krb5kdc: starting...
W20260810 14:57:25.388463 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:25 dist-test-slave-0b58 krb5kdc[21515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:57:25.522936 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36519
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:25.611866 21531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:25.612138 21531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:25.612166 21531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:25.613497 21531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:25.613554 21531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:25.613569 21531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:25.613582 21531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:25.613595 21531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:25.615279 21531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:25.615569 21531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:25.615826 21531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:25.621110 21537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:25.621104 21539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:25.621193 21531 server_base.cc:1034] running on GCE node
W20260810 14:57:25.621166 21536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:25.621596 21531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:25.621924 21531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:25.623121 21531 hybrid_clock.cc:648] HybridClock initialized: now 1786373845623108 us; error 53 us; skew 500 ppm
Aug 10 14:57:25 dist-test-slave-0b58 krb5kdc[21515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373845, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:25.626021 21531 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:57:25.626539 21531 webserver.cc:460] Webserver started at http://127.0.0.1:39911/ using document root <none> and password file <none>
I20260810 14:57:25.626749 21531 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:25.626793 21531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:25.626891 21531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:25.627992 21531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "af41ba04a9134509bf0806e2e8a66867"
format_stamp: "Formatted at 2026-08-10 14:57:25 on dist-test-slave-0b58"
server_key: "9aab4e6868399aa64195b42aea14e176"
server_key_iv: "75622f28eb9d6c69e1a3c09e25cc3897"
server_key_version: "encryptionkey@0"
I20260810 14:57:25.628403 21531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "af41ba04a9134509bf0806e2e8a66867"
format_stamp: "Formatted at 2026-08-10 14:57:25 on dist-test-slave-0b58"
server_key: "9aab4e6868399aa64195b42aea14e176"
server_key_iv: "75622f28eb9d6c69e1a3c09e25cc3897"
server_key_version: "encryptionkey@0"
I20260810 14:57:25.629855 21531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:25.630733 21547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:25.631067 21531 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:25.631183 21531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "af41ba04a9134509bf0806e2e8a66867"
format_stamp: "Formatted at 2026-08-10 14:57:25 on dist-test-slave-0b58"
server_key: "9aab4e6868399aa64195b42aea14e176"
server_key_iv: "75622f28eb9d6c69e1a3c09e25cc3897"
server_key_version: "encryptionkey@0"
I20260810 14:57:25.631264 21531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:25.660621 21531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:25.663017 21531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:25.663214 21531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:25.667930 21531 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36519
I20260810 14:57:25.667975 21610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36519 every 8 connection(s)
I20260810 14:57:25.668399 21531 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:25.669284 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 21531
I20260810 14:57:25.669423 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:25.669620 21611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:25.669667 31017 external_mini_cluster.cc:1428] Setting key b08164424213b08c6bbf9e00c03ecb5c
I20260810 14:57:25.673465 21611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867: Bootstrap starting.
I20260810 14:57:25.674157 21611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:25.674502 21611 log.cc:824] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:25.675369 21611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867: No bootstrap required, opened a new log
I20260810 14:57:25.675853 21615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:25.670278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:59461 (local address 127.0.0.1:36519)
0810 14:57:25.670477 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:25.670493 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:25.670524 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:57:25.671111 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:25.671120 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:25.671186 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:25.671314 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:25.671614 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:25.672641 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:25.673744 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:25.674154 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:25.674281 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:25.675124 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:25.675128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:25.675128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:25.675318 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:25.675350 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:25.675361 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:25.675466 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:25.675567 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 14:57:25.675612 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
I20260810 14:57:25.677359 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:25.677307 21611 raft_consensus.cc:348] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af41ba04a9134509bf0806e2e8a66867" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36519 } }
I20260810 14:57:25.677500 21611 raft_consensus.cc:374] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:25.677539 21611 raft_consensus.cc:729] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af41ba04a9134509bf0806e2e8a66867, State: Initialized, Role: FOLLOWER
I20260810 14:57:25.677654 21611 consensus_queue.cc:260] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af41ba04a9134509bf0806e2e8a66867" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36519 } }
I20260810 14:57:25.677726 21611 raft_consensus.cc:388] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:25.677752 21611 raft_consensus.cc:482] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:25.677789 21611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:25.678613 21611 raft_consensus.cc:504] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af41ba04a9134509bf0806e2e8a66867" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36519 } }
I20260810 14:57:25.678732 21611 leader_election.cc:302] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af41ba04a9134509bf0806e2e8a66867; no voters:
I20260810 14:57:25.678934 21611 leader_election.cc:288] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:25.679013 21621 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:25.679183 21621 raft_consensus.cc:686] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [term 1 LEADER]: Becoming Leader. State: Replica: af41ba04a9134509bf0806e2e8a66867, State: Running, Role: LEADER
I20260810 14:57:25.679281 21611 sys_catalog.cc:564] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:25.679315 21621 consensus_queue.cc:237] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af41ba04a9134509bf0806e2e8a66867" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36519 } }
I20260810 14:57:25.679795 21623 sys_catalog.cc:455] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af41ba04a9134509bf0806e2e8a66867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af41ba04a9134509bf0806e2e8a66867" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36519 } } }
I20260810 14:57:25.679888 21623 sys_catalog.cc:458] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:25.680094 21624 sys_catalog.cc:455] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af41ba04a9134509bf0806e2e8a66867. Latest consensus state: current_term: 1 leader_uuid: "af41ba04a9134509bf0806e2e8a66867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af41ba04a9134509bf0806e2e8a66867" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36519 } } }
I20260810 14:57:25.680130 21632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:25.680972 21632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:25.681595 21624 sys_catalog.cc:458] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:25.682538 21632 catalog_manager.cc:1347] Generated new cluster ID: 6f128f1f9ed846e3a899041c08d53084
I20260810 14:57:25.682595 21632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:25.683929 21615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:25.679018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:45891 (local address 127.0.0.1:36519)
0810 14:57:25.679061 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:25.679063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:25.680507 (+ 1444us) server_negotiation.cc:407] Connection header received
0810 14:57:25.680621 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:25.680623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:25.680648 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:25.680684 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:25.680985 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:25.681497 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:25.682565 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:25.682851 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:25.682931 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:25.683673 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:25.683674 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:25.683675 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:25.683680 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:25.683696 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:25.683701 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:25.683770 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:25.683837 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 14:57:25.683849 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 14:57:25.687750 21632 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:25.688544 21632 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:25.693943 21632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af41ba04a9134509bf0806e2e8a66867: Generated new TSK 0
I20260810 14:57:25.694166 21632 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:25.698120 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 21531
W20260810 14:57:25.705119 21513 connection.cc:535] client connection to 127.0.0.1:36519 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36519): Connection reset by peer (error 104)
2026-08-10T14:57:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:25 dist-test-slave-0b58 krb5kdc[21648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:25 dist-test-slave-0b58 krb5kdc[21648](info): set up 2 sockets
Aug 10 14:57:25 dist-test-slave-0b58 krb5kdc[21648](info): commencing operation
krb5kdc: starting...
W20260810 14:57:26.801476 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:26 dist-test-slave-0b58 krb5kdc[21648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:57:26.944155 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45957
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:27.033954 21662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:27.034268 21662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:27.034317 21662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:27.035676 21662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:27.035764 21662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:27.035789 21662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:27.035811 21662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:27.035840 21662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:27.037693 21662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:27.038039 21662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:27.038313 21662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:27.043903 21670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:27.043906 21668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:27.043903 21667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:27.044344 21662 server_base.cc:1034] running on GCE node
I20260810 14:57:27.044646 21662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:27.045001 21662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:27.046257 21662 hybrid_clock.cc:648] HybridClock initialized: now 1786373847046235 us; error 50 us; skew 500 ppm
Aug 10 14:57:27 dist-test-slave-0b58 krb5kdc[21648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:27.049141 21662 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:57:27.049705 21662 webserver.cc:460] Webserver started at http://127.0.0.1:42819/ using document root <none> and password file <none>
I20260810 14:57:27.049983 21662 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:27.050067 21662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:27.050247 21662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:27.051445 21662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "900e6641e7264dcd86daf4fdbb625fc6"
format_stamp: "Formatted at 2026-08-10 14:57:27 on dist-test-slave-0b58"
server_key: "d5687b20590e1ffee61c6c9d8d4db0fc"
server_key_iv: "38555b7d19fcef439ac70aba88bbeed1"
server_key_version: "encryptionkey@0"
I20260810 14:57:27.051865 21662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "900e6641e7264dcd86daf4fdbb625fc6"
format_stamp: "Formatted at 2026-08-10 14:57:27 on dist-test-slave-0b58"
server_key: "d5687b20590e1ffee61c6c9d8d4db0fc"
server_key_iv: "38555b7d19fcef439ac70aba88bbeed1"
server_key_version: "encryptionkey@0"
I20260810 14:57:27.053650 21662 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260810 14:57:27.054622 21678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:27.054857 21662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:57:27.054980 21662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "900e6641e7264dcd86daf4fdbb625fc6"
format_stamp: "Formatted at 2026-08-10 14:57:27 on dist-test-slave-0b58"
server_key: "d5687b20590e1ffee61c6c9d8d4db0fc"
server_key_iv: "38555b7d19fcef439ac70aba88bbeed1"
server_key_version: "encryptionkey@0"
I20260810 14:57:27.055069 21662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:27.058821 21662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:27.061391 21662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:27.061605 21662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:27.065980 21662 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45957
I20260810 14:57:27.066038 21734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45957 every 8 connection(s)
I20260810 14:57:27.066466 21662 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:27.067492 21735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:27.070091 21735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6: Bootstrap starting.
I20260810 14:57:27.070708 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 21662
I20260810 14:57:27.070859 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:27.070924 21735 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:27.071072 31017 external_mini_cluster.cc:1428] Setting key ff42510a732435d4cc3646b7a7679ad6
I20260810 14:57:27.071303 21735 log.cc:824] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:27.072515 21735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6: No bootstrap required, opened a new log
I20260810 14:57:27.074352 21739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:27.071687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:52749 (local address 127.0.0.1:45957)
0810 14:57:27.071906 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:27.071918 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:27.071964 (+ 46us) server_negotiation.cc:407] Connection header received
0810 14:57:27.072649 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:27.072660 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:27.072701 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:27.072877 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:27.073700 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:27.073703 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:27.073704 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:27.073708 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:27.073772 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:27.073785 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:27.073860 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:27.073947 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 14:57:27.074113 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
I20260810 14:57:27.075218 21735 raft_consensus.cc:348] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900e6641e7264dcd86daf4fdbb625fc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45957 } }
I20260810 14:57:27.075404 21735 raft_consensus.cc:374] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:27.075456 21735 raft_consensus.cc:729] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 900e6641e7264dcd86daf4fdbb625fc6, State: Initialized, Role: FOLLOWER
I20260810 14:57:27.075554 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:27.075620 21735 consensus_queue.cc:260] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900e6641e7264dcd86daf4fdbb625fc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45957 } }
I20260810 14:57:27.075721 21735 raft_consensus.cc:388] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:27.075762 21735 raft_consensus.cc:482] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:27.075809 21735 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:27.076761 21735 raft_consensus.cc:504] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900e6641e7264dcd86daf4fdbb625fc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45957 } }
I20260810 14:57:27.076907 21735 leader_election.cc:302] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 900e6641e7264dcd86daf4fdbb625fc6; no voters:
I20260810 14:57:27.077112 21735 leader_election.cc:288] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:27.077312 21751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:27.077435 21735 sys_catalog.cc:564] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:27.077593 21751 raft_consensus.cc:686] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [term 1 LEADER]: Becoming Leader. State: Replica: 900e6641e7264dcd86daf4fdbb625fc6, State: Running, Role: LEADER
I20260810 14:57:27.077762 21751 consensus_queue.cc:237] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900e6641e7264dcd86daf4fdbb625fc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45957 } }
I20260810 14:57:27.078442 21754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 900e6641e7264dcd86daf4fdbb625fc6. Latest consensus state: current_term: 1 leader_uuid: "900e6641e7264dcd86daf4fdbb625fc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900e6641e7264dcd86daf4fdbb625fc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45957 } } }
I20260810 14:57:27.078553 21754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:27.078975 21739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:27.077351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:34969 (local address 127.0.0.1:45957)
0810 14:57:27.077388 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:27.077391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:27.077585 (+ 194us) server_negotiation.cc:407] Connection header received
0810 14:57:27.077715 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:27.077716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:27.077722 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:27.077753 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:27.078555 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:27.078557 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:27.078557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:27.078562 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:27.078579 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:27.078584 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:27.078640 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:27.078889 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 14:57:27.078905 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 14:57:27.079165 21750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "900e6641e7264dcd86daf4fdbb625fc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900e6641e7264dcd86daf4fdbb625fc6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45957 } } }
W20260810 14:57:27.079583 21769 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260810 14:57:27.079855 21750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6 [sys.catalog]: This master's current role is: LEADER
W20260810 14:57:27.079864 21769 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:57:27.079957 21763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:27.080166 21763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:27.081976 21763 catalog_manager.cc:1347] Generated new cluster ID: 1cebf0d033a04c36ba41d0647c4cbe87
I20260810 14:57:27.082067 21763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:27.096418 21763 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:27.096652 21763 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:27.100886 21763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 900e6641e7264dcd86daf4fdbb625fc6: Generated new TSK 0
I20260810 14:57:27.101155 21763 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:27.110266 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 21662
2026-08-10T14:57:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1411 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:27 dist-test-slave-0b58 krb5kdc[21776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:27 dist-test-slave-0b58 krb5kdc[21776](info): set up 2 sockets
Aug 10 14:57:27 dist-test-slave-0b58 krb5kdc[21776](info): commencing operation
krb5kdc: starting...
W20260810 14:57:28.218408 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:28 dist-test-slave-0b58 krb5kdc[21776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:57:28.372112 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34103
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:28.460881 21791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:28.461328 21791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:28.461432 21791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:28.462845 21791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:28.462932 21791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:28.462958 21791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:28.462982 21791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:28.463042 21791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:28.465389 21791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34103
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:28.465737 21791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:28.465997 21791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:28.471787 21797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:28.471904 21791 server_base.cc:1034] running on GCE node
W20260810 14:57:28.471740 21800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:28.471740 21798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:28.472326 21791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:28.472748 21791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:28.473973 21791 hybrid_clock.cc:648] HybridClock initialized: now 1786373848473963 us; error 41 us; skew 500 ppm
Aug 10 14:57:28 dist-test-slave-0b58 krb5kdc[21776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373848, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:28.477097 21791 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:57:28.477813 21791 webserver.cc:460] Webserver started at http://127.0.0.1:39545/ using document root <none> and password file <none>
I20260810 14:57:28.478106 21791 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:28.478184 21791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:28.478478 21791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:28.479727 21791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "b538602b33dc4ee88e02f6d45f030a47"
format_stamp: "Formatted at 2026-08-10 14:57:28 on dist-test-slave-0b58"
server_key: "b0214c664abfc8a9c1c422388285e8d9"
server_key_iv: "fd69b2b9091ed496df318115bde08c37"
server_key_version: "encryptionkey@0"
I20260810 14:57:28.480499 21791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "b538602b33dc4ee88e02f6d45f030a47"
format_stamp: "Formatted at 2026-08-10 14:57:28 on dist-test-slave-0b58"
server_key: "b0214c664abfc8a9c1c422388285e8d9"
server_key_iv: "fd69b2b9091ed496df318115bde08c37"
server_key_version: "encryptionkey@0"
I20260810 14:57:28.482378 21791 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 14:57:28.483402 21809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:28.483680 21791 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:28.483817 21791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "b538602b33dc4ee88e02f6d45f030a47"
format_stamp: "Formatted at 2026-08-10 14:57:28 on dist-test-slave-0b58"
server_key: "b0214c664abfc8a9c1c422388285e8d9"
server_key_iv: "fd69b2b9091ed496df318115bde08c37"
server_key_version: "encryptionkey@0"
I20260810 14:57:28.483912 21791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:28.513447 21791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:28.515601 21791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:28.515787 21791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:28.520083 21791 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34103
I20260810 14:57:28.520123 21869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34103 every 8 connection(s)
I20260810 14:57:28.520555 21791 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:28.521520 21870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:28.525133 21870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47: Bootstrap starting.
I20260810 14:57:28.525951 21870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:28.526315 21870 log.cc:824] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:28.527354 21870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47: No bootstrap required, opened a new log
I20260810 14:57:28.529011 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 21791
I20260810 14:57:28.529135 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:28.529356 31017 external_mini_cluster.cc:1428] Setting key 9a0b664c6095e283ebee0812a8afc2f3
I20260810 14:57:28.529548 21870 raft_consensus.cc:348] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b538602b33dc4ee88e02f6d45f030a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34103 } }
I20260810 14:57:28.529791 21870 raft_consensus.cc:374] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:28.529866 21870 raft_consensus.cc:729] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b538602b33dc4ee88e02f6d45f030a47, State: Initialized, Role: FOLLOWER
I20260810 14:57:28.530021 21870 consensus_queue.cc:260] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b538602b33dc4ee88e02f6d45f030a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34103 } }
I20260810 14:57:28.530125 21870 raft_consensus.cc:388] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:28.530172 21870 raft_consensus.cc:482] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:28.530231 21870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:28.531209 21870 raft_consensus.cc:504] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b538602b33dc4ee88e02f6d45f030a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34103 } }
I20260810 14:57:28.531343 21870 leader_election.cc:302] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b538602b33dc4ee88e02f6d45f030a47; no voters:
I20260810 14:57:28.531558 21870 leader_election.cc:288] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:28.531616 21878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:28.531781 21878 raft_consensus.cc:686] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [term 1 LEADER]: Becoming Leader. State: Replica: b538602b33dc4ee88e02f6d45f030a47, State: Running, Role: LEADER
I20260810 14:57:28.531852 21870 sys_catalog.cc:564] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:28.531946 21878 consensus_queue.cc:237] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b538602b33dc4ee88e02f6d45f030a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34103 } }
I20260810 14:57:28.532400 21878 sys_catalog.cc:455] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b538602b33dc4ee88e02f6d45f030a47. Latest consensus state: current_term: 1 leader_uuid: "b538602b33dc4ee88e02f6d45f030a47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b538602b33dc4ee88e02f6d45f030a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34103 } } }
I20260810 14:57:28.532493 21878 sys_catalog.cc:458] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:28.532413 21879 sys_catalog.cc:455] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b538602b33dc4ee88e02f6d45f030a47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b538602b33dc4ee88e02f6d45f030a47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34103 } } }
I20260810 14:57:28.532722 21879 sys_catalog.cc:458] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47 [sys.catalog]: This master's current role is: LEADER
W20260810 14:57:28.533953 21895 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:57:28.534060 21895 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:57:28.534862 21887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:28.535085 21887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:28.536813 21887 catalog_manager.cc:1347] Generated new cluster ID: 2b6b2296271c4e82a78f878477169b2a
I20260810 14:57:28.536868 21887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 14:57:28 dist-test-slave-0b58 krb5kdc[21776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 14:57:28.541399 21875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:28.530070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:51317 (local address 127.0.0.1:34103)
0810 14:57:28.530335 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:28.530344 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:28.530380 (+ 36us) server_negotiation.cc:407] Connection header received
0810 14:57:28.530491 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:28.530497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:28.530567 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:28.530718 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:28.531557 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:28.532967 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:28.534151 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:28.534595 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:28.534682 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:28.537718 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:28.537723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:28.537723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:28.537737 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:28.539550 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:28.540019 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:28.540027 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:28.540027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:28.540138 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:28.540453 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:28.540454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:28.540454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:28.540737 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:28.540929 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:28.541061 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 14:57:28.541179 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":121,"threads_started":1}
I20260810 14:57:28.542486 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:28.545728 21887 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:28.546489 21887 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:28.550807 21875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:28.543934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:33163 (local address 127.0.0.1:34103)
0810 14:57:28.543977 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:28.543979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:28.544103 (+ 124us) server_negotiation.cc:407] Connection header received
0810 14:57:28.544182 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:28.544184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:28.544194 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:28.544250 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:28.544838 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:28.545442 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:28.546325 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:28.546579 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:28.546646 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:28.547997 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:28.547999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:28.547999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:28.548007 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:28.549524 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:28.549929 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:28.549931 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:28.549931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:28.549992 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:28.550228 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:28.550229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:28.550229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:28.550351 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:28.550484 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:28.550598 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 14:57:28.550670 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 14:57:28.561076 21887 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b538602b33dc4ee88e02f6d45f030a47: Generated new TSK 0
I20260810 14:57:28.561334 21887 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:28.565814 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 21791
W20260810 14:57:28.573077 21774 connection.cc:535] client connection to 127.0.0.1:34103 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34103): Connection reset by peer (error 104)
2026-08-10T14:57:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:28 dist-test-slave-0b58 krb5kdc[21906](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:28 dist-test-slave-0b58 krb5kdc[21906](info): set up 2 sockets
Aug 10 14:57:28 dist-test-slave-0b58 krb5kdc[21906](info): commencing operation
krb5kdc: starting...
W20260810 14:57:29.668095 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:29 dist-test-slave-0b58 krb5kdc[21906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:57:29.801200 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40145
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:29.885128 21923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:29.885394 21923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:29.885433 21923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:29.886766 21923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:29.886837 21923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:29.886864 21923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:29.886896 21923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:29.886925 21923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:29.888584 21923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40145
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:29.888897 21923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:29.889148 21923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:29.894317 21933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:29.894369 21930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:29.894374 21931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:29.894635 21923 server_base.cc:1034] running on GCE node
I20260810 14:57:29.894975 21923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:29.895274 21923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:29.896433 21923 hybrid_clock.cc:648] HybridClock initialized: now 1786373849896415 us; error 28 us; skew 500 ppm
Aug 10 14:57:29 dist-test-slave-0b58 krb5kdc[21906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373849, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:29.899026 21923 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:57:29.899576 21923 webserver.cc:460] Webserver started at http://127.0.0.1:44173/ using document root <none> and password file <none>
I20260810 14:57:29.899822 21923 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:29.899870 21923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:29.900086 21923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:29.901378 21923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "c9167bcd1efd4215b5b040070809fbf0"
format_stamp: "Formatted at 2026-08-10 14:57:29 on dist-test-slave-0b58"
server_key: "749546d324525c35b80ef400a48e9d72"
server_key_iv: "abd073d7d6dc88932c04674fb3cb322f"
server_key_version: "encryptionkey@0"
I20260810 14:57:29.901774 21923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "c9167bcd1efd4215b5b040070809fbf0"
format_stamp: "Formatted at 2026-08-10 14:57:29 on dist-test-slave-0b58"
server_key: "749546d324525c35b80ef400a48e9d72"
server_key_iv: "abd073d7d6dc88932c04674fb3cb322f"
server_key_version: "encryptionkey@0"
I20260810 14:57:29.903160 21923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:57:29.903935 21944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:29.904158 21923 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:29.904297 21923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "c9167bcd1efd4215b5b040070809fbf0"
format_stamp: "Formatted at 2026-08-10 14:57:29 on dist-test-slave-0b58"
server_key: "749546d324525c35b80ef400a48e9d72"
server_key_iv: "abd073d7d6dc88932c04674fb3cb322f"
server_key_version: "encryptionkey@0"
I20260810 14:57:29.904394 21923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:29.916124 21923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:29.918418 21923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:29.918603 21923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:29.923147 21923 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40145
I20260810 14:57:29.923192 21999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40145 every 8 connection(s)
I20260810 14:57:29.923602 21923 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:29.924566 22000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:29.926890 22000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0: Bootstrap starting.
I20260810 14:57:29.927042 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 21923
I20260810 14:57:29.927184 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:29.927392 31017 external_mini_cluster.cc:1428] Setting key 5ebf6cf90e78761f9224de2a8ea4b758
I20260810 14:57:29.927650 22000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:29.928035 22000 log.cc:824] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:29.928911 22000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0: No bootstrap required, opened a new log
I20260810 14:57:29.931329 22000 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9167bcd1efd4215b5b040070809fbf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40145 } }
I20260810 14:57:29.931479 22000 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:29.931501 22000 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9167bcd1efd4215b5b040070809fbf0, State: Initialized, Role: FOLLOWER
I20260810 14:57:29.931638 22000 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9167bcd1efd4215b5b040070809fbf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40145 } }
I20260810 14:57:29.931713 22000 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:29.931768 22000 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:29.931828 22000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:29.932497 22000 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9167bcd1efd4215b5b040070809fbf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40145 } }
I20260810 14:57:29.932619 22000 leader_election.cc:302] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9167bcd1efd4215b5b040070809fbf0; no voters:
I20260810 14:57:29.932825 22000 leader_election.cc:288] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:29.932843 22008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:29.933032 22008 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [term 1 LEADER]: Becoming Leader. State: Replica: c9167bcd1efd4215b5b040070809fbf0, State: Running, Role: LEADER
I20260810 14:57:29.933198 22000 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:29.933169 22008 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9167bcd1efd4215b5b040070809fbf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40145 } }
I20260810 14:57:29.933658 22009 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9167bcd1efd4215b5b040070809fbf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9167bcd1efd4215b5b040070809fbf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40145 } } }
I20260810 14:57:29.933820 22009 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:29.934262 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:29.928007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:58609 (local address 127.0.0.1:40145)
0810 14:57:29.928298 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:29.928315 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:29.928353 (+ 38us) server_negotiation.cc:407] Connection header received
0810 14:57:29.928949 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:29.928959 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:29.929030 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:29.929190 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:29.929394 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:29.930848 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:29.931829 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:29.932416 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:29.932544 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:29.933376 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:29.933380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:29.933380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:29.933618 (+ 238us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:29.933670 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:29.933687 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:29.933791 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:29.934001 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 14:57:29.934031 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":99,"threads_started":1}
I20260810 14:57:29.934580 22017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:29.934733 22010 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9167bcd1efd4215b5b040070809fbf0. Latest consensus state: current_term: 1 leader_uuid: "c9167bcd1efd4215b5b040070809fbf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9167bcd1efd4215b5b040070809fbf0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40145 } } }
I20260810 14:57:29.934890 22010 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:29.935251 22017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:29.935572 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:29.936990 22017 catalog_manager.cc:1347] Generated new cluster ID: a973bac8140c4decb8fb105efdfd01f4
I20260810 14:57:29.937050 22017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:29.940224 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:29.937368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:45301 (local address 127.0.0.1:40145)
0810 14:57:29.937405 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:29.937407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:29.937415 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:57:29.937435 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:29.937436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:29.937445 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:29.937499 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:29.937688 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:29.938183 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:29.938918 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:29.939152 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:29.939229 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:29.939956 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:29.939957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:29.939958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:29.939963 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:29.939979 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:29.939983 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:29.940067 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:29.940132 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 14:57:29.940142 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 14:57:29.948839 22017 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:29.949570 22017 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:29.954617 22017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9167bcd1efd4215b5b040070809fbf0: Generated new TSK 0
I20260810 14:57:29.954841 22017 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:29.957545 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 21923
W20260810 14:57:29.964242 21904 connection.cc:535] client connection to 127.0.0.1:40145 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40145): Connection reset by peer (error 104)
2026-08-10T14:57:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1391 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:30 dist-test-slave-0b58 krb5kdc[22034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:30 dist-test-slave-0b58 krb5kdc[22034](info): set up 2 sockets
Aug 10 14:57:30 dist-test-slave-0b58 krb5kdc[22034](info): commencing operation
krb5kdc: starting...
W20260810 14:57:31.051349 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:31 dist-test-slave-0b58 krb5kdc[22034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:57:31.187763 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40863
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:31.272764 22050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:31.273046 22050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:31.273089 22050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:31.274399 22050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:31.274480 22050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:31.274504 22050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:31.274526 22050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:31.274555 22050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:31.276182 22050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40863
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:31.276521 22050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:31.276773 22050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 14:57:31.282619 22050 server_base.cc:1034] running on GCE node
W20260810 14:57:31.282572 22057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:31.282521 22060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:31.282521 22056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:31.283177 22050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:31.283582 22050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:31.284783 22050 hybrid_clock.cc:648] HybridClock initialized: now 1786373851284745 us; error 52 us; skew 500 ppm
Aug 10 14:57:31 dist-test-slave-0b58 krb5kdc[22034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373851, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:31.287575 22050 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:57:31.288080 22050 webserver.cc:460] Webserver started at http://127.0.0.1:33777/ using document root <none> and password file <none>
I20260810 14:57:31.288417 22050 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:31.288494 22050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:31.288661 22050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:31.289783 22050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "25c2c2770fa0467da2b855ba9f6fe4c6"
format_stamp: "Formatted at 2026-08-10 14:57:31 on dist-test-slave-0b58"
server_key: "57308ade93c05ac57c80d6a77f4ba739"
server_key_iv: "4958e5a4d821d0abf5e5c7d6129deadd"
server_key_version: "encryptionkey@0"
I20260810 14:57:31.290170 22050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "25c2c2770fa0467da2b855ba9f6fe4c6"
format_stamp: "Formatted at 2026-08-10 14:57:31 on dist-test-slave-0b58"
server_key: "57308ade93c05ac57c80d6a77f4ba739"
server_key_iv: "4958e5a4d821d0abf5e5c7d6129deadd"
server_key_version: "encryptionkey@0"
I20260810 14:57:31.291683 22050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:57:31.292606 22067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:31.292797 22050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:31.292918 22050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "25c2c2770fa0467da2b855ba9f6fe4c6"
format_stamp: "Formatted at 2026-08-10 14:57:31 on dist-test-slave-0b58"
server_key: "57308ade93c05ac57c80d6a77f4ba739"
server_key_iv: "4958e5a4d821d0abf5e5c7d6129deadd"
server_key_version: "encryptionkey@0"
I20260810 14:57:31.293001 22050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:31.296495 22050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:31.298669 22050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:31.298853 22050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:31.303100 22050 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40863
I20260810 14:57:31.303179 22123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40863 every 8 connection(s)
I20260810 14:57:31.303629 22050 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:31.304065 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 22050
I20260810 14:57:31.304191 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:31.304428 31017 external_mini_cluster.cc:1428] Setting key 7d1aa0f4b9ea70ef56aafc8d55618d13
I20260810 14:57:31.304916 22124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:31.307317 22124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6: Bootstrap starting.
I20260810 14:57:31.307942 22124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:31.308240 22124 log.cc:824] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:31.309080 22124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6: No bootstrap required, opened a new log
I20260810 14:57:31.310541 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:31.305074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:40747 (local address 127.0.0.1:40863)
0810 14:57:31.305289 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:31.305298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:31.308581 (+ 3283us) server_negotiation.cc:407] Connection header received
0810 14:57:31.309192 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:31.309208 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:31.309243 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:31.309378 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:31.310059 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:31.310062 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:31.310062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:31.310066 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:31.310105 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:31.310116 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:31.310186 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:31.310272 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 14:57:31.310333 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":115,"threads_started":1}
I20260810 14:57:31.312136 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:31.312093 22124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c2c2770fa0467da2b855ba9f6fe4c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40863 } }
I20260810 14:57:31.312377 22124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:31.312429 22124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25c2c2770fa0467da2b855ba9f6fe4c6, State: Initialized, Role: FOLLOWER
I20260810 14:57:31.312585 22124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c2c2770fa0467da2b855ba9f6fe4c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40863 } }
I20260810 14:57:31.312668 22124 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:31.312693 22124 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:31.312762 22124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:31.313413 22124 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c2c2770fa0467da2b855ba9f6fe4c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40863 } }
I20260810 14:57:31.313549 22124 leader_election.cc:302] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25c2c2770fa0467da2b855ba9f6fe4c6; no voters:
I20260810 14:57:31.313745 22124 leader_election.cc:288] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:31.313793 22135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:31.313969 22135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [term 1 LEADER]: Becoming Leader. State: Replica: 25c2c2770fa0467da2b855ba9f6fe4c6, State: Running, Role: LEADER
I20260810 14:57:31.314079 22124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:31.314390 22135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c2c2770fa0467da2b855ba9f6fe4c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40863 } }
I20260810 14:57:31.315236 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:31.314100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.101:40931 (local address 127.0.0.1:40863)
0810 14:57:31.314166 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:31.314168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:31.314178 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:57:31.314203 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:31.314204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:31.314210 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:31.314275 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:31.314946 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:31.314948 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:31.314949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:31.314954 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:31.314972 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:31.314976 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:31.315012 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:31.315143 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 14:57:31.315161 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 14:57:31.315296 22137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25c2c2770fa0467da2b855ba9f6fe4c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c2c2770fa0467da2b855ba9f6fe4c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40863 } } }
I20260810 14:57:31.315403 22137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:31.315639 22139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25c2c2770fa0467da2b855ba9f6fe4c6. Latest consensus state: current_term: 1 leader_uuid: "25c2c2770fa0467da2b855ba9f6fe4c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c2c2770fa0467da2b855ba9f6fe4c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40863 } } }
I20260810 14:57:31.315687 22146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:31.315709 22139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:31.316401 22146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:31.318069 22146 catalog_manager.cc:1347] Generated new cluster ID: ed032a987b614836b8259967286262b0
I20260810 14:57:31.318132 22146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:31.328459 22146 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:31.328667 22146 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:31.339521 22146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25c2c2770fa0467da2b855ba9f6fe4c6: Generated new TSK 0
I20260810 14:57:31.339777 22146 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:31.343607 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 22050
2026-08-10T14:57:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:31 dist-test-slave-0b58 krb5kdc[22159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:31 dist-test-slave-0b58 krb5kdc[22159](info): set up 2 sockets
Aug 10 14:57:31 dist-test-slave-0b58 krb5kdc[22159](info): commencing operation
krb5kdc: starting...
W20260810 14:57:32.431362 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:32 dist-test-slave-0b58 krb5kdc[22159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:57:32.560086 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:35633
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:42123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:35633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:32.647897 22173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:32.648123 22173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:32.648146 22173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:32.649370 22173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:32.649420 22173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:32.649434 22173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:32.649447 22173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:32.649461 22173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:32.651046 22173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:42123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:35633
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:35633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:32.651335 22173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:32.651588 22173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:32.656944 22181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:32.656976 22173 server_base.cc:1034] running on GCE node
W20260810 14:57:32.656944 22179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:32.656944 22178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:32.657562 22173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:32.657878 22173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:32.659047 22173 hybrid_clock.cc:648] HybridClock initialized: now 1786373852659026 us; error 40 us; skew 500 ppm
Aug 10 14:57:32 dist-test-slave-0b58 krb5kdc[22159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373852, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:32.666874 22173 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:57:32.667300 22173 webserver.cc:460] Webserver started at http://127.30.74.126:35453/ using document root <none> and password file <none>
I20260810 14:57:32.667511 22173 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:32.667553 22173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:32.667654 22173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:32.668774 22173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "48a26025f2c24dedb8383ceed854cb1d"
format_stamp: "Formatted at 2026-08-10 14:57:32 on dist-test-slave-0b58"
server_key: "93fa48782df9fcb833ccbed061277fb2"
server_key_iv: "a5ec23fe6473c4aed5f47950fdfc3b03"
server_key_version: "encryptionkey@0"
I20260810 14:57:32.669085 22173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "48a26025f2c24dedb8383ceed854cb1d"
format_stamp: "Formatted at 2026-08-10 14:57:32 on dist-test-slave-0b58"
server_key: "93fa48782df9fcb833ccbed061277fb2"
server_key_iv: "a5ec23fe6473c4aed5f47950fdfc3b03"
server_key_version: "encryptionkey@0"
I20260810 14:57:32.670476 22173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:57:32.671223 22191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:32.671422 22173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:32.671516 22173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "48a26025f2c24dedb8383ceed854cb1d"
format_stamp: "Formatted at 2026-08-10 14:57:32 on dist-test-slave-0b58"
server_key: "93fa48782df9fcb833ccbed061277fb2"
server_key_iv: "a5ec23fe6473c4aed5f47950fdfc3b03"
server_key_version: "encryptionkey@0"
I20260810 14:57:32.671576 22173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:32.693450 22173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:32.695858 22173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:32.695978 22173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:32.699934 22173 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:35633
I20260810 14:57:32.700007 22248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:35633 every 8 connection(s)
I20260810 14:57:32.700398 22173 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:32.701361 22249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:32.704341 22249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d: Bootstrap starting.
I20260810 14:57:32.704967 22249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:32.705277 22249 log.cc:824] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:32.706019 22249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d: No bootstrap required, opened a new log
I20260810 14:57:32.706976 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 22173
I20260810 14:57:32.707108 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:32.707317 31017 external_mini_cluster.cc:1428] Setting key b9d0625207d3d69219e694fa4b0d5598
I20260810 14:57:32.708101 22249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a26025f2c24dedb8383ceed854cb1d" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 35633 } }
I20260810 14:57:32.708308 22249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:32.708351 22249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48a26025f2c24dedb8383ceed854cb1d, State: Initialized, Role: FOLLOWER
I20260810 14:57:32.708484 22249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a26025f2c24dedb8383ceed854cb1d" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 35633 } }
I20260810 14:57:32.708573 22249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:32.708606 22249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:32.708729 22249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:32.709805 22249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a26025f2c24dedb8383ceed854cb1d" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 35633 } }
I20260810 14:57:32.709918 22249 leader_election.cc:302] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48a26025f2c24dedb8383ceed854cb1d; no voters:
I20260810 14:57:32.710081 22249 leader_election.cc:288] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:32.710142 22255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:32.710326 22255 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [term 1 LEADER]: Becoming Leader. State: Replica: 48a26025f2c24dedb8383ceed854cb1d, State: Running, Role: LEADER
I20260810 14:57:32.710356 22249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:32.710423 22255 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a26025f2c24dedb8383ceed854cb1d" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 35633 } }
I20260810 14:57:32.710845 22256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48a26025f2c24dedb8383ceed854cb1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a26025f2c24dedb8383ceed854cb1d" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 35633 } } }
I20260810 14:57:32.710940 22256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:32.712088 22255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48a26025f2c24dedb8383ceed854cb1d. Latest consensus state: current_term: 1 leader_uuid: "48a26025f2c24dedb8383ceed854cb1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a26025f2c24dedb8383ceed854cb1d" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 35633 } } }
I20260810 14:57:32.712229 22255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d [sys.catalog]: This master's current role is: LEADER
W20260810 14:57:32.712368 22275 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:57:32.712438 22275 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:57:32.712535 22267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:32.712718 22267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:32.714653 22267 catalog_manager.cc:1347] Generated new cluster ID: cab66282699c415684339fe23b3c2176
I20260810 14:57:32.714720 22267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 14:57:32 dist-test-slave-0b58 krb5kdc[22159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:32.720755 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:32.707877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51365 (local address 127.30.74.126:35633)
0810 14:57:32.708078 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:32.708088 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:32.708128 (+ 40us) server_negotiation.cc:407] Connection header received
0810 14:57:32.708302 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:32.708310 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:32.708383 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:32.708533 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:32.709343 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:32.710771 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:32.711874 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:32.712556 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:32.712671 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:32.715879 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:32.715883 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:32.715883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:32.715896 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:32.718043 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:32.718571 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:32.718575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:32.718575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:32.718709 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:32.718971 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:32.718972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:32.718972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:32.719319 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:32.719903 (+ 584us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:32.720424 (+ 521us) server_negotiation.cc:299] Negotiation successful
0810 14:57:32.720538 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
I20260810 14:57:32.721926 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:32.722303 22267 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:32.723081 22267 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:32.728597 22267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48a26025f2c24dedb8383ceed854cb1d: Generated new TSK 0
I20260810 14:57:32.728785 22267 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:32.730983 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:32.723436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49509 (local address 127.30.74.126:35633)
0810 14:57:32.723475 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:32.723477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:32.723669 (+ 192us) server_negotiation.cc:407] Connection header received
0810 14:57:32.723699 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:32.723700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:32.723711 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:32.723780 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:32.724392 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:32.725219 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:32.726235 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:32.726583 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:32.726655 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:32.728057 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:32.728059 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:32.728059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:32.728066 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:32.729871 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:32.730227 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:32.730228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:32.730229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:32.730290 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:32.730500 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:32.730501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:32.730501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:32.730629 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:32.730742 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:32.730828 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 14:57:32.730895 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 14:57:32.732864 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 22173
W20260810 14:57:32.739820 22157 connection.cc:535] client connection to 127.30.74.126:35633 recv error: Network error: failed to read from TLS socket (remote: 127.30.74.126:35633): Connection reset by peer (error 104)
2026-08-10T14:57:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1389 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:32 dist-test-slave-0b58 krb5kdc[22286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:32 dist-test-slave-0b58 krb5kdc[22286](info): set up 2 sockets
Aug 10 14:57:32 dist-test-slave-0b58 krb5kdc[22286](info): commencing operation
krb5kdc: starting...
W20260810 14:57:33.812438 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:33 dist-test-slave-0b58 krb5kdc[22286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:57:33.940996 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:46431
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:46431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:34.023998 22305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:34.024217 22305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:34.024241 22305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:34.025450 22305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:34.025497 22305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:34.025511 22305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:34.025524 22305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:34.025537 22305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:34.027073 22305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:46431
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:46431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:34.027340 22305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:34.027576 22305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:34.032635 22310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:34.032651 22311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:34.032900 22305 server_base.cc:1034] running on GCE node
W20260810 14:57:34.032658 22313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:34.033286 22305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:34.033555 22305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:34.034698 22305 hybrid_clock.cc:648] HybridClock initialized: now 1786373854034685 us; error 32 us; skew 500 ppm
Aug 10 14:57:34 dist-test-slave-0b58 krb5kdc[22286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373854, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:34.037389 22305 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:57:34.037845 22305 webserver.cc:460] Webserver started at http://127.30.74.126:36443/ using document root <none> and password file <none>
I20260810 14:57:34.038110 22305 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:34.038187 22305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:34.038334 22305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:34.039413 22305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "a5f298ca3b384732a3fa2b428d59e584"
format_stamp: "Formatted at 2026-08-10 14:57:34 on dist-test-slave-0b58"
server_key: "ee1864055fecb932f6ce7b173354b769"
server_key_iv: "8a0cd26f64eee66ce5c266df3b78f082"
server_key_version: "encryptionkey@0"
I20260810 14:57:34.039788 22305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "a5f298ca3b384732a3fa2b428d59e584"
format_stamp: "Formatted at 2026-08-10 14:57:34 on dist-test-slave-0b58"
server_key: "ee1864055fecb932f6ce7b173354b769"
server_key_iv: "8a0cd26f64eee66ce5c266df3b78f082"
server_key_version: "encryptionkey@0"
I20260810 14:57:34.041234 22305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:57:34.041980 22321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:34.042151 22305 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:34.042289 22305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "a5f298ca3b384732a3fa2b428d59e584"
format_stamp: "Formatted at 2026-08-10 14:57:34 on dist-test-slave-0b58"
server_key: "ee1864055fecb932f6ce7b173354b769"
server_key_iv: "8a0cd26f64eee66ce5c266df3b78f082"
server_key_version: "encryptionkey@0"
I20260810 14:57:34.042363 22305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:34.064941 22305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:34.067502 22305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:34.067688 22305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:34.071659 22305 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:46431
I20260810 14:57:34.071754 22383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:46431 every 8 connection(s)
I20260810 14:57:34.072117 22305 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:34.073016 22384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:34.075269 22384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584: Bootstrap starting.
I20260810 14:57:34.075815 22384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:34.076050 22384 log.cc:824] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:34.076777 22384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584: No bootstrap required, opened a new log
I20260810 14:57:34.076848 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 22305
I20260810 14:57:34.076992 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:34.077185 31017 external_mini_cluster.cc:1428] Setting key c4324e2f75c69318dce4513d197e9d43
I20260810 14:57:34.078794 22384 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5f298ca3b384732a3fa2b428d59e584" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 46431 } }
I20260810 14:57:34.078974 22384 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:34.079015 22384 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f298ca3b384732a3fa2b428d59e584, State: Initialized, Role: FOLLOWER
I20260810 14:57:34.079213 22384 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5f298ca3b384732a3fa2b428d59e584" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 46431 } }
I20260810 14:57:34.079334 22384 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:34.079381 22384 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:34.079437 22384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:34.079994 22384 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5f298ca3b384732a3fa2b428d59e584" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 46431 } }
I20260810 14:57:34.080113 22384 leader_election.cc:302] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5f298ca3b384732a3fa2b428d59e584; no voters:
I20260810 14:57:34.080354 22384 leader_election.cc:288] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:34.080380 22390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:34.080513 22390 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [term 1 LEADER]: Becoming Leader. State: Replica: a5f298ca3b384732a3fa2b428d59e584, State: Running, Role: LEADER
I20260810 14:57:34.080631 22384 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:34.080658 22390 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5f298ca3b384732a3fa2b428d59e584" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 46431 } }
I20260810 14:57:34.081050 22390 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5f298ca3b384732a3fa2b428d59e584. Latest consensus state: current_term: 1 leader_uuid: "a5f298ca3b384732a3fa2b428d59e584" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5f298ca3b384732a3fa2b428d59e584" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 46431 } } }
I20260810 14:57:34.081069 22391 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5f298ca3b384732a3fa2b428d59e584" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5f298ca3b384732a3fa2b428d59e584" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 46431 } } }
I20260810 14:57:34.081204 22390 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:34.081424 22391 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:34.081579 22400 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:34.082103 22400 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:34.084203 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:34.077747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47809 (local address 127.30.74.126:46431)
0810 14:57:34.077934 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:34.077941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:34.077967 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:57:34.078777 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:34.078783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:34.078834 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:34.078949 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:34.079267 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:34.080713 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:34.081909 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:34.082502 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:34.082618 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:34.083464 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:34.083468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:34.083469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:34.083655 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:34.083699 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:34.083710 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:34.083814 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:34.083943 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 14:57:34.083976 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260810 14:57:34.084275 22400 catalog_manager.cc:1347] Generated new cluster ID: 1b26aa4fadcf44bfb55d90356db0262f
I20260810 14:57:34.084324 22400 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:34.085036 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:34.090140 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:34.086508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41257 (local address 127.30.74.126:46431)
0810 14:57:34.086569 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:34.086571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:34.086597 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:57:34.086668 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:34.086669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:34.086680 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:34.086760 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:34.086901 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:34.087629 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:34.088653 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:34.089058 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:34.089134 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:34.089869 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:34.089871 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:34.089871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:34.089876 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:34.089894 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:34.089899 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:34.089965 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:34.090045 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 14:57:34.090059 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 14:57:34.091338 22400 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:34.092036 22400 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:34.100317 22400 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5f298ca3b384732a3fa2b428d59e584: Generated new TSK 0
I20260810 14:57:34.100555 22400 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:34.103482 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 22305
W20260810 14:57:34.110538 22284 connection.cc:535] client connection to 127.30.74.126:46431 recv error: Network error: failed to read from TLS socket (remote: 127.30.74.126:46431): Connection reset by peer (error 104)
2026-08-10T14:57:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1370 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:34 dist-test-slave-0b58 krb5kdc[22421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:34 dist-test-slave-0b58 krb5kdc[22421](info): set up 2 sockets
Aug 10 14:57:34 dist-test-slave-0b58 krb5kdc[22421](info): commencing operation
krb5kdc: starting...
W20260810 14:57:35.191491 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:35 dist-test-slave-0b58 krb5kdc[22421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:57:35.323172 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:33331
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:33331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:35.411725 22434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:35.411958 22434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:35.411983 22434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:35.413290 22434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:35.413341 22434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:35.413355 22434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:35.413367 22434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:35.413380 22434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:35.414944 22434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:33331
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:33331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:35.415166 22434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:35.415364 22434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:35.420569 22443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:35.420521 22442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:35.420540 22445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:35.420694 22434 server_base.cc:1034] running on GCE node
I20260810 14:57:35.421051 22434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:35.421429 22434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:35.422600 22434 hybrid_clock.cc:648] HybridClock initialized: now 1786373855422595 us; error 39 us; skew 500 ppm
Aug 10 14:57:35 dist-test-slave-0b58 krb5kdc[22421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373855, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:35.425262 22434 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:57:35.425940 22434 webserver.cc:460] Webserver started at http://127.30.74.126:44163/ using document root <none> and password file <none>
I20260810 14:57:35.426290 22434 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:35.426374 22434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:35.426518 22434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:35.427623 22434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "31a19606509b483fa450ea1b372d1422"
format_stamp: "Formatted at 2026-08-10 14:57:35 on dist-test-slave-0b58"
server_key: "f494948f3b6cf63ca8a570ac9bc36e56"
server_key_iv: "9f087c0e342de85d7e5faba09e6e25db"
server_key_version: "encryptionkey@0"
I20260810 14:57:35.428007 22434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "31a19606509b483fa450ea1b372d1422"
format_stamp: "Formatted at 2026-08-10 14:57:35 on dist-test-slave-0b58"
server_key: "f494948f3b6cf63ca8a570ac9bc36e56"
server_key_iv: "9f087c0e342de85d7e5faba09e6e25db"
server_key_version: "encryptionkey@0"
I20260810 14:57:35.429489 22434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:57:35.430325 22452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:35.430541 22434 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:35.430644 22434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "31a19606509b483fa450ea1b372d1422"
format_stamp: "Formatted at 2026-08-10 14:57:35 on dist-test-slave-0b58"
server_key: "f494948f3b6cf63ca8a570ac9bc36e56"
server_key_iv: "9f087c0e342de85d7e5faba09e6e25db"
server_key_version: "encryptionkey@0"
I20260810 14:57:35.430779 22434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:35.434278 22434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:35.436676 22434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:35.436853 22434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:35.441133 22434 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:33331
I20260810 14:57:35.441174 22508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:33331 every 8 connection(s)
I20260810 14:57:35.441524 22434 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:35.442596 22510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:35.445137 22510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422: Bootstrap starting.
I20260810 14:57:35.445791 22510 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:35.446102 22510 log.cc:824] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:35.446914 22510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422: No bootstrap required, opened a new log
I20260810 14:57:35.448691 22510 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a19606509b483fa450ea1b372d1422" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 33331 } }
I20260810 14:57:35.448846 22510 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:35.448870 22510 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a19606509b483fa450ea1b372d1422, State: Initialized, Role: FOLLOWER
I20260810 14:57:35.449025 22510 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a19606509b483fa450ea1b372d1422" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 33331 } }
I20260810 14:57:35.449115 22510 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:35.449139 22510 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:35.449168 22510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:35.449707 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 22434
I20260810 14:57:35.449838 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:35.450101 31017 external_mini_cluster.cc:1428] Setting key debebea51146dc16828f5a86b1e9447c
I20260810 14:57:35.450488 22510 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a19606509b483fa450ea1b372d1422" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 33331 } }
I20260810 14:57:35.450604 22510 leader_election.cc:302] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31a19606509b483fa450ea1b372d1422; no voters:
I20260810 14:57:35.450825 22510 leader_election.cc:288] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:35.450843 22515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:35.451030 22515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [term 1 LEADER]: Becoming Leader. State: Replica: 31a19606509b483fa450ea1b372d1422, State: Running, Role: LEADER
I20260810 14:57:35.451174 22515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a19606509b483fa450ea1b372d1422" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 33331 } }
I20260810 14:57:35.451181 22510 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:35.451969 22517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31a19606509b483fa450ea1b372d1422" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a19606509b483fa450ea1b372d1422" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 33331 } } }
I20260810 14:57:35.452471 22517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:35.452368 22516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31a19606509b483fa450ea1b372d1422. Latest consensus state: current_term: 1 leader_uuid: "31a19606509b483fa450ea1b372d1422" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a19606509b483fa450ea1b372d1422" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 33331 } } }
I20260810 14:57:35.452744 22530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:35.452744 22516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:35.452996 22514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:35.450634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42581 (local address 127.30.74.126:33331)
0810 14:57:35.450862 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:35.450873 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:35.450914 (+ 41us) server_negotiation.cc:407] Connection header received
0810 14:57:35.451071 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:35.451080 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:35.451111 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:35.451622 (+ 511us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:35.452144 (+ 522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:35.452147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:35.452148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:35.452152 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:35.452189 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:35.452202 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:35.452556 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:35.452591 (+ 35us) server_negotiation.cc:299] Negotiation successful
0810 14:57:35.452641 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":105,"threads_started":1}
I20260810 14:57:35.453526 22530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:35.454142 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:35.456801 22530 catalog_manager.cc:1347] Generated new cluster ID: 2d79a1bf3b544e1eae68645e0e39aaa6
I20260810 14:57:35.456858 22530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:35.457202 22514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:35.456133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58053 (local address 127.30.74.126:33331)
0810 14:57:35.456196 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:35.456198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:35.456209 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:57:35.456232 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:35.456232 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:35.456238 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:35.456338 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:35.456984 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:35.456985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:35.456985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:35.456990 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:35.457005 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:35.457009 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:35.457056 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:35.457118 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 14:57:35.457134 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 14:57:35.471140 22530 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:35.471344 22530 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:35.474447 22530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31a19606509b483fa450ea1b372d1422: Generated new TSK 0
I20260810 14:57:35.474656 22530 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:35.481426 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 22434
2026-08-10T14:57:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1377 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:35 dist-test-slave-0b58 krb5kdc[22544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:35 dist-test-slave-0b58 krb5kdc[22544](info): set up 2 sockets
Aug 10 14:57:35 dist-test-slave-0b58 krb5kdc[22544](info): commencing operation
krb5kdc: starting...
W20260810 14:57:36.571991 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:36 dist-test-slave-0b58 krb5kdc[22544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:57:36.699604 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:41061
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:41677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:41061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:36.786415 22561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:36.786702 22561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:36.786747 22561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:36.788077 22561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:36.788174 22561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:36.788204 22561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:36.788237 22561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:36.788306 22561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:36.789963 22561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:41677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:41061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:41061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:36.790289 22561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:36.790550 22561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:36.795876 22569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:36.795888 22567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:36.795886 22566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:36.795922 22561 server_base.cc:1034] running on GCE node
I20260810 14:57:36.796321 22561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:36.796615 22561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:36.797797 22561 hybrid_clock.cc:648] HybridClock initialized: now 1786373856797785 us; error 50 us; skew 500 ppm
Aug 10 14:57:36 dist-test-slave-0b58 krb5kdc[22544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373856, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:36.800554 22561 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:57:36.801041 22561 webserver.cc:460] Webserver started at http://127.30.74.126:36251/ using document root <none> and password file <none>
I20260810 14:57:36.801298 22561 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:36.801347 22561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:36.801493 22561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:36.802657 22561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "779220f63d6548349e203d0da4898fb9"
format_stamp: "Formatted at 2026-08-10 14:57:36 on dist-test-slave-0b58"
server_key: "5cb6b00040f077002b44978eb633c18c"
server_key_iv: "387989d8bc3d5012324ffbe75068aefa"
server_key_version: "encryptionkey@0"
I20260810 14:57:36.803051 22561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "779220f63d6548349e203d0da4898fb9"
format_stamp: "Formatted at 2026-08-10 14:57:36 on dist-test-slave-0b58"
server_key: "5cb6b00040f077002b44978eb633c18c"
server_key_iv: "387989d8bc3d5012324ffbe75068aefa"
server_key_version: "encryptionkey@0"
I20260810 14:57:36.804569 22561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:57:36.805441 22578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:36.805634 22561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:36.805749 22561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "779220f63d6548349e203d0da4898fb9"
format_stamp: "Formatted at 2026-08-10 14:57:36 on dist-test-slave-0b58"
server_key: "5cb6b00040f077002b44978eb633c18c"
server_key_iv: "387989d8bc3d5012324ffbe75068aefa"
server_key_version: "encryptionkey@0"
I20260810 14:57:36.805840 22561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:36.816495 22561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:36.818944 22561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:36.819123 22561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:36.823853 22561 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:41061
I20260810 14:57:36.823906 22634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:41061 every 8 connection(s)
I20260810 14:57:36.824348 22561 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:36.825318 22635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:36.825742 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 22561
I20260810 14:57:36.825881 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:36.826056 31017 external_mini_cluster.cc:1428] Setting key 769c9a2a6ada5d2a016ebda49c19eba6
I20260810 14:57:36.828526 22635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9: Bootstrap starting.
I20260810 14:57:36.829180 22635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:36.829558 22635 log.cc:824] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:36.830427 22635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9: No bootstrap required, opened a new log
I20260810 14:57:36.832542 22635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779220f63d6548349e203d0da4898fb9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41061 } }
I20260810 14:57:36.832696 22635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:36.832721 22635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 779220f63d6548349e203d0da4898fb9, State: Initialized, Role: FOLLOWER
I20260810 14:57:36.832875 22635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779220f63d6548349e203d0da4898fb9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41061 } }
I20260810 14:57:36.832943 22635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:36.833004 22635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:36.833076 22635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [term 0 FOLLOWER]: Advancing to term 1
Aug 10 14:57:36 dist-test-slave-0b58 krb5kdc[22544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:36.833741 22635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779220f63d6548349e203d0da4898fb9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41061 } }
I20260810 14:57:36.833880 22635 leader_election.cc:302] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 779220f63d6548349e203d0da4898fb9; no voters:
I20260810 14:57:36.834105 22635 leader_election.cc:288] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:36.834141 22640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:36.834312 22640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [term 1 LEADER]: Becoming Leader. State: Replica: 779220f63d6548349e203d0da4898fb9, State: Running, Role: LEADER
I20260810 14:57:36.834453 22640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779220f63d6548349e203d0da4898fb9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41061 } }
I20260810 14:57:36.834486 22635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:36.834960 22642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 779220f63d6548349e203d0da4898fb9. Latest consensus state: current_term: 1 leader_uuid: "779220f63d6548349e203d0da4898fb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779220f63d6548349e203d0da4898fb9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41061 } } }
I20260810 14:57:36.835052 22642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:36.835263 22641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "779220f63d6548349e203d0da4898fb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "779220f63d6548349e203d0da4898fb9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 41061 } } }
I20260810 14:57:36.835418 22641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:36.835865 22645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:36.836491 22645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:36.838392 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:36.826693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54979 (local address 127.30.74.126:41061)
0810 14:57:36.826871 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:36.826880 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:36.826907 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:57:36.827320 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:36.827326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:36.827375 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:36.827498 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:36.828415 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:36.829849 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:36.830868 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:36.831334 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:36.831480 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:36.834228 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:36.834233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:36.834233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:36.834248 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:36.836602 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:36.837021 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:36.837025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:36.837025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:36.837138 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:36.837386 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:36.837387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:36.837388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:36.837688 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:36.837846 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:36.837972 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 14:57:36.838120 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260810 14:57:36.838864 22645 catalog_manager.cc:1347] Generated new cluster ID: 6e8b6a80eaeb47c9a8affc43e6b29ebe
I20260810 14:57:36.838915 22645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:36.839474 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:36.845247 22645 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:36.845944 22645 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:36.852231 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:36.840979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53739 (local address 127.30.74.126:41061)
0810 14:57:36.841013 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:36.841016 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:36.844549 (+ 3533us) server_negotiation.cc:407] Connection header received
0810 14:57:36.844618 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:36.844620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:36.844633 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:36.844718 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:36.845555 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:36.846246 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:36.847312 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:36.847622 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:36.847689 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:36.849279 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:36.849280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:36.849281 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:36.849289 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:36.850946 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:36.851385 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:36.851386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:36.851387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:36.851448 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:36.851665 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:36.851666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:36.851666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:36.851793 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:36.851902 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:36.852031 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 14:57:36.852131 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260810 14:57:36.858796 22645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 779220f63d6548349e203d0da4898fb9: Generated new TSK 0
I20260810 14:57:36.859035 22645 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:36.862797 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 22561
2026-08-10T14:57:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1381 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:36 dist-test-slave-0b58 krb5kdc[22667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:36 dist-test-slave-0b58 krb5kdc[22667](info): set up 2 sockets
Aug 10 14:57:36 dist-test-slave-0b58 krb5kdc[22667](info): commencing operation
krb5kdc: starting...
W20260810 14:57:37.959359 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:38 dist-test-slave-0b58 krb5kdc[22667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:57:38.088367 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:44071
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:37565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:44071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:38.174154 22682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:38.174387 22682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:38.174412 22682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:38.175561 22682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:38.175604 22682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:38.175618 22682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:38.175631 22682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:38.175643 22682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:38.177158 22682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:37565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:44071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:44071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:38.177385 22682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:38.177577 22682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:38.182566 22690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:38.182657 22688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:38.182574 22687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:38.182749 22682 server_base.cc:1034] running on GCE node
I20260810 14:57:38.183056 22682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:38.183372 22682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:38.184549 22682 hybrid_clock.cc:648] HybridClock initialized: now 1786373858184527 us; error 58 us; skew 500 ppm
Aug 10 14:57:38 dist-test-slave-0b58 krb5kdc[22667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373858, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:38.191793 22682 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:57:38.192399 22682 webserver.cc:460] Webserver started at http://127.30.74.126:35071/ using document root <none> and password file <none>
I20260810 14:57:38.192651 22682 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:38.192736 22682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:38.192906 22682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:38.194056 22682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "ef517a1a08a04bd5960a3b2e48bdf85a"
format_stamp: "Formatted at 2026-08-10 14:57:38 on dist-test-slave-0b58"
server_key: "46581c66f3e46307bb759042053c690d"
server_key_iv: "8172752a176a3c0df90f7c8380bc8f86"
server_key_version: "encryptionkey@0"
I20260810 14:57:38.194459 22682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "ef517a1a08a04bd5960a3b2e48bdf85a"
format_stamp: "Formatted at 2026-08-10 14:57:38 on dist-test-slave-0b58"
server_key: "46581c66f3e46307bb759042053c690d"
server_key_iv: "8172752a176a3c0df90f7c8380bc8f86"
server_key_version: "encryptionkey@0"
I20260810 14:57:38.195940 22682 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:57:38.196956 22697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:38.197191 22682 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 14:57:38.197309 22682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "ef517a1a08a04bd5960a3b2e48bdf85a"
format_stamp: "Formatted at 2026-08-10 14:57:38 on dist-test-slave-0b58"
server_key: "46581c66f3e46307bb759042053c690d"
server_key_iv: "8172752a176a3c0df90f7c8380bc8f86"
server_key_version: "encryptionkey@0"
I20260810 14:57:38.197405 22682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:38.221863 22682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:38.225287 22682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:38.225482 22682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:38.230731 22682 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:44071
I20260810 14:57:38.230775 22754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:44071 every 8 connection(s)
I20260810 14:57:38.231161 22682 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:38.232048 22755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:38.234457 22755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a: Bootstrap starting.
I20260810 14:57:38.234476 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 22682
I20260810 14:57:38.234607 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:38.234809 31017 external_mini_cluster.cc:1428] Setting key 6c72364cd9ce492d915fba682f164327
I20260810 14:57:38.235212 22755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:38.235795 22755 log.cc:824] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:38.236685 22755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a: No bootstrap required, opened a new log
I20260810 14:57:38.238910 22755 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef517a1a08a04bd5960a3b2e48bdf85a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44071 } }
I20260810 14:57:38.239097 22755 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:38.239148 22755 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef517a1a08a04bd5960a3b2e48bdf85a, State: Initialized, Role: FOLLOWER
I20260810 14:57:38.239291 22755 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef517a1a08a04bd5960a3b2e48bdf85a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44071 } }
I20260810 14:57:38.239394 22755 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:38.239440 22755 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:38.239488 22755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:38.240298 22755 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef517a1a08a04bd5960a3b2e48bdf85a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44071 } }
I20260810 14:57:38.240418 22755 leader_election.cc:302] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef517a1a08a04bd5960a3b2e48bdf85a; no voters:
I20260810 14:57:38.240633 22755 leader_election.cc:288] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:38.240658 22761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:38.240792 22761 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [term 1 LEADER]: Becoming Leader. State: Replica: ef517a1a08a04bd5960a3b2e48bdf85a, State: Running, Role: LEADER
I20260810 14:57:38.240916 22755 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:38.240933 22761 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef517a1a08a04bd5960a3b2e48bdf85a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44071 } }
I20260810 14:57:38.241448 22763 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef517a1a08a04bd5960a3b2e48bdf85a. Latest consensus state: current_term: 1 leader_uuid: "ef517a1a08a04bd5960a3b2e48bdf85a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef517a1a08a04bd5960a3b2e48bdf85a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44071 } } }
I20260810 14:57:38.241433 22762 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef517a1a08a04bd5960a3b2e48bdf85a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef517a1a08a04bd5960a3b2e48bdf85a" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 44071 } } }
I20260810 14:57:38.241667 22762 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:38.241426 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:38.235396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53521 (local address 127.30.74.126:44071)
0810 14:57:38.235586 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:38.235597 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:38.235637 (+ 40us) server_negotiation.cc:407] Connection header received
0810 14:57:38.236311 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:38.236321 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:38.236391 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:38.236543 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:38.236884 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:38.237830 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:38.239046 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:38.239625 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:38.239717 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:38.240461 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:38.240465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:38.240465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:38.240664 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:38.240727 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:38.240743 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:38.241103 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:38.241138 (+ 35us) server_negotiation.cc:299] Negotiation successful
0810 14:57:38.241210 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":69,"threads_started":1}
I20260810 14:57:38.241652 22763 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:38.242218 22771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:38.242695 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:38.242729 22771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:38.244681 22771 catalog_manager.cc:1347] Generated new cluster ID: d1d1903d09b34af0a582c0cc5e2440b8
I20260810 14:57:38.244854 22771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:38.247664 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:38.244683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34075 (local address 127.30.74.126:44071)
0810 14:57:38.244725 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:38.244727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:38.244817 (+ 90us) server_negotiation.cc:407] Connection header received
0810 14:57:38.244925 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:38.244926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:38.244938 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:38.244996 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:38.245156 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:38.245641 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:38.246370 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:38.246618 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:38.246700 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:38.247423 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:38.247424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:38.247425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:38.247429 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:38.247444 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:38.247448 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:38.247492 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:38.247547 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 14:57:38.247573 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:57:38.266507 22771 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:38.267081 22771 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:38.281220 22771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef517a1a08a04bd5960a3b2e48bdf85a: Generated new TSK 0
I20260810 14:57:38.281466 22771 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:38.291266 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 22682
2026-08-10T14:57:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:38 dist-test-slave-0b58 krb5kdc[22789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:38 dist-test-slave-0b58 krb5kdc[22789](info): set up 2 sockets
Aug 10 14:57:38 dist-test-slave-0b58 krb5kdc[22789](info): commencing operation
krb5kdc: starting...
W20260810 14:57:39.374799 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:39 dist-test-slave-0b58 krb5kdc[22789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:57:39.503717 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:40195
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:35797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:40195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:39.588459 22802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:39.588728 22802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:39.588769 22802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:39.589965 22802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:39.590050 22802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:39.590075 22802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:39.590106 22802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:39.590137 22802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:39.591691 22802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:35797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:40195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:40195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:39.591997 22802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:39.592248 22802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:39.597327 22811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:39.597362 22809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:39.597362 22808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:39.597599 22802 server_base.cc:1034] running on GCE node
I20260810 14:57:39.597885 22802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:39.598155 22802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:39.599321 22802 hybrid_clock.cc:648] HybridClock initialized: now 1786373859599298 us; error 37 us; skew 500 ppm
Aug 10 14:57:39 dist-test-slave-0b58 krb5kdc[22789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373859, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:39.601953 22802 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:57:39.602439 22802 webserver.cc:460] Webserver started at http://127.30.74.126:44423/ using document root <none> and password file <none>
I20260810 14:57:39.602638 22802 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:39.602679 22802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:39.602773 22802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:39.603824 22802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "df93abafb0bc4a9688c42edbe3b9a1a9"
format_stamp: "Formatted at 2026-08-10 14:57:39 on dist-test-slave-0b58"
server_key: "0ac028eec82f9cec38fe8b775e063c93"
server_key_iv: "d7c8836f4f43788fb096a4827033815f"
server_key_version: "encryptionkey@0"
I20260810 14:57:39.604138 22802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "df93abafb0bc4a9688c42edbe3b9a1a9"
format_stamp: "Formatted at 2026-08-10 14:57:39 on dist-test-slave-0b58"
server_key: "0ac028eec82f9cec38fe8b775e063c93"
server_key_iv: "d7c8836f4f43788fb096a4827033815f"
server_key_version: "encryptionkey@0"
I20260810 14:57:39.605554 22802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:57:39.606309 22818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:39.606491 22802 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:57:39.606608 22802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "df93abafb0bc4a9688c42edbe3b9a1a9"
format_stamp: "Formatted at 2026-08-10 14:57:39 on dist-test-slave-0b58"
server_key: "0ac028eec82f9cec38fe8b775e063c93"
server_key_iv: "d7c8836f4f43788fb096a4827033815f"
server_key_version: "encryptionkey@0"
I20260810 14:57:39.606691 22802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:39.610216 22802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:39.612192 22802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:39.612396 22802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:39.616580 22802 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:40195
I20260810 14:57:39.616631 22875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:40195 every 8 connection(s)
I20260810 14:57:39.616986 22802 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:39.617961 22876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:39.619674 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 22802
I20260810 14:57:39.619813 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:39.620020 31017 external_mini_cluster.cc:1428] Setting key 20ea02c4e205b6c612d4a15d742c16b9
I20260810 14:57:39.621146 22876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9: Bootstrap starting.
I20260810 14:57:39.622349 22876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:39.622690 22876 log.cc:824] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:39.622927 22879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:39.620622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40773 (local address 127.30.74.126:40195)
0810 14:57:39.620812 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:39.620822 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:39.620855 (+ 33us) server_negotiation.cc:407] Connection header received
0810 14:57:39.621443 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:39.621453 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:39.621491 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:39.621645 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:39.622358 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:39.622362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:39.622362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:39.622366 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:39.622436 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:39.622451 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:39.622528 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:39.622660 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 14:57:39.622710 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260810 14:57:39.624106 22876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9: No bootstrap required, opened a new log
I20260810 14:57:39.624322 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:39.627040 22879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:39.625948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56527 (local address 127.30.74.126:40195)
0810 14:57:39.625990 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:39.625992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:39.626071 (+ 79us) server_negotiation.cc:407] Connection header received
0810 14:57:39.626162 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:39.626162 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:39.626166 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:39.626212 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:39.626794 (+ 582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:39.626795 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:39.626795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:57:39.626799 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:39.626813 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:57:39.626816 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:39.626856 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:39.626916 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 14:57:39.626926 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 14:57:39.626950 22876 raft_consensus.cc:348] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df93abafb0bc4a9688c42edbe3b9a1a9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 40195 } }
I20260810 14:57:39.627143 22876 raft_consensus.cc:374] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:57:39.627184 22876 raft_consensus.cc:729] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df93abafb0bc4a9688c42edbe3b9a1a9, State: Initialized, Role: FOLLOWER
I20260810 14:57:39.627315 22876 consensus_queue.cc:260] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df93abafb0bc4a9688c42edbe3b9a1a9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 40195 } }
I20260810 14:57:39.627394 22876 raft_consensus.cc:388] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:57:39.627416 22876 raft_consensus.cc:482] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:57:39.627486 22876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:39.628058 22876 raft_consensus.cc:504] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df93abafb0bc4a9688c42edbe3b9a1a9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 40195 } }
I20260810 14:57:39.628180 22876 leader_election.cc:302] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df93abafb0bc4a9688c42edbe3b9a1a9; no voters:
I20260810 14:57:39.628376 22876 leader_election.cc:288] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:57:39.628414 22886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:39.628588 22886 raft_consensus.cc:686] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [term 1 LEADER]: Becoming Leader. State: Replica: df93abafb0bc4a9688c42edbe3b9a1a9, State: Running, Role: LEADER
I20260810 14:57:39.628684 22876 sys_catalog.cc:564] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:39.628710 22886 consensus_queue.cc:237] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df93abafb0bc4a9688c42edbe3b9a1a9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 40195 } }
I20260810 14:57:39.629125 22888 sys_catalog.cc:455] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df93abafb0bc4a9688c42edbe3b9a1a9. Latest consensus state: current_term: 1 leader_uuid: "df93abafb0bc4a9688c42edbe3b9a1a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df93abafb0bc4a9688c42edbe3b9a1a9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 40195 } } }
I20260810 14:57:39.629249 22888 sys_catalog.cc:458] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:39.629444 22895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:39.629112 22887 sys_catalog.cc:455] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df93abafb0bc4a9688c42edbe3b9a1a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df93abafb0bc4a9688c42edbe3b9a1a9" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 40195 } } }
I20260810 14:57:39.629601 22887 sys_catalog.cc:458] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:39.629976 22895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:39.637476 22895 catalog_manager.cc:1347] Generated new cluster ID: 9390339e68734cd0bbcb167cddf374bd
I20260810 14:57:39.637550 22895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:39.647323 22895 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:39.647538 22895 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:39.653079 22895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df93abafb0bc4a9688c42edbe3b9a1a9: Generated new TSK 0
I20260810 14:57:39.653314 22895 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:39.662058 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 22802
2026-08-10T14:57:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1371 ms)
[----------] 18 tests from AuthTokenIssuingTest (25447 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:39 dist-test-slave-0b58 krb5kdc[22910](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:39 dist-test-slave-0b58 krb5kdc[22910](info): set up 2 sockets
Aug 10 14:57:39 dist-test-slave-0b58 krb5kdc[22910](info): commencing operation
krb5kdc: starting...
W20260810 14:57:40.769111 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:40 dist-test-slave-0b58 krb5kdc[22910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:57:40.899408 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:34437
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:44743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:34437,127.30.74.125:38409,127.30.74.124:42341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:40.987306 22924 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:57:40.987587 22924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:40.987627 22924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:40.987668 22924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:40.988917 22924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:40.988991 22924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:40.989017 22924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:40.989050 22924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:40.989078 22924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:40.990671 22924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:44743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:34437,127.30.74.125:38409,127.30.74.124:42341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:34437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:40.990971 22924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:40.991220 22924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:40.996482 22932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:40.996562 22924 server_base.cc:1034] running on GCE node
W20260810 14:57:40.996486 22929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:40.996482 22930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:40.996990 22924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:40.997326 22924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:40.998517 22924 hybrid_clock.cc:648] HybridClock initialized: now 1786373860998494 us; error 46 us; skew 500 ppm
Aug 10 14:57:40 dist-test-slave-0b58 krb5kdc[22910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373860, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:41.001286 22924 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:57:41.001835 22924 webserver.cc:460] Webserver started at http://127.30.74.126:45665/ using document root <none> and password file <none>
I20260810 14:57:41.002049 22924 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:41.002094 22924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:41.002192 22924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:41.003484 22924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "1eade91c12f9499889b5e79c565d4f76"
format_stamp: "Formatted at 2026-08-10 14:57:40 on dist-test-slave-0b58"
server_key: "aeef2a5580be9c6da9d204ed07d14356"
server_key_iv: "13e584e263d1998c2c1aa83d8893cf29"
server_key_version: "encryptionkey@0"
I20260810 14:57:41.003839 22924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "1eade91c12f9499889b5e79c565d4f76"
format_stamp: "Formatted at 2026-08-10 14:57:40 on dist-test-slave-0b58"
server_key: "aeef2a5580be9c6da9d204ed07d14356"
server_key_iv: "13e584e263d1998c2c1aa83d8893cf29"
server_key_version: "encryptionkey@0"
I20260810 14:57:41.005735 22924 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260810 14:57:41.006574 22941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:41.006843 22924 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:41.006951 22924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "1eade91c12f9499889b5e79c565d4f76"
format_stamp: "Formatted at 2026-08-10 14:57:40 on dist-test-slave-0b58"
server_key: "aeef2a5580be9c6da9d204ed07d14356"
server_key_iv: "13e584e263d1998c2c1aa83d8893cf29"
server_key_version: "encryptionkey@0"
I20260810 14:57:41.007036 22924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:41.021560 22924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:41.023876 22924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:41.024053 22924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:41.028388 22924 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:34437
I20260810 14:57:41.028445 22997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:34437 every 8 connection(s)
I20260810 14:57:41.028827 22924 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:41.029858 22998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:41.031450 22998 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:57:41.035974 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 22924
I20260810 14:57:41.036132 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:41.036386 31017 external_mini_cluster.cc:1428] Setting key 84c5007faa94b64783f82ec72dfb697c
Aug 10 14:57:41 dist-test-slave-0b58 krb5kdc[22910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373860, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:41.042203 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.032196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:34437 (local address 127.0.0.1:33878)
0810 14:57:41.032406 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.032450 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:41.032951 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:41.033249 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:41.033252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:41.033539 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:41.033692 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.033697 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.034739 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.034739 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.035420 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.035422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.035990 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.035992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.036059 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.036711 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:41.036720 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:41.038417 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:41.040603 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.040610 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.040612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.040814 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.041116 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.041116 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.041117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.041207 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.041721 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:41.041728 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:41.041792 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 14:57:41.041879 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 14:57:41.042016 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":128,"threads_started":1}
I20260810 14:57:41.042284 23001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.032232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33878 (local address 127.30.74.126:34437)
0810 14:57:41.032530 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.032534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.032565 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:57:41.033114 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.033116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.033136 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.033244 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.033793 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.034619 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.035511 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.035866 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.036015 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.038520 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:41.038530 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:41.038531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:41.038540 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:41.040520 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.040901 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.040904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.040905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.041037 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.041280 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.041281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.041281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.041610 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:41.041722 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.041945 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.042109 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":271,"thread_start_us":245,"threads_started":1}
I20260810 14:57:41.043081 22998 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:57:41.043474 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.043326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:38409 (local address 127.0.0.1:48900)
0810 14:57:41.043384 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.043441 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:38409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 14:57:41.043637 22945 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.74.125:38409: connect: Connection refused (error 111)
I20260810 14:57:41.044119 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.044011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:38409 (local address 127.0.0.1:48910)
0810 14:57:41.044061 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.044096 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:38409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 14:57:41.044299 22998 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.125:38409: Network error: Client connection negotiation failed: client connection to 127.30.74.125:38409: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.30.74.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.125.keytab.
Entry for principal kudu/127.30.74.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.125.keytab.
I20260810 14:57:41.065506 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.125:38409
--webserver_interface=127.30.74.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.125.keytab
--principal=kudu/127.30.74.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:44743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:34437,127.30.74.125:38409,127.30.74.124:42341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:57:41.093571 22998 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } attempt: 1
I20260810 14:57:41.094259 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.094046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:38409 (local address 127.0.0.1:48912)
0810 14:57:41.094138 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.094223 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:38409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 14:57:41.094776 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.094628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:38409 (local address 127.0.0.1:48920)
0810 14:57:41.094704 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.094749 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:38409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 14:57:41.094916 22998 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.125:38409: Network error: Client connection negotiation failed: client connection to 127.30.74.125:38409: connect: Connection refused (error 111)
W20260810 14:57:41.151257 23005 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:57:41.151494 23005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:41.151512 23005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:41.151535 23005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:41.152689 23005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:41.152731 23005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:41.152745 23005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:41.152760 23005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:41.152771 23005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:41.154232 23005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:44743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:34437,127.30.74.125:38409,127.30.74.124:42341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.125.keytab
--principal=kudu/127.30.74.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.125:38409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.74.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:41.154439 23005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:41.154621 23005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 14:57:41.159960 23005 server_base.cc:1034] running on GCE node
W20260810 14:57:41.159961 23012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:41.159998 23011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:41.159978 23014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:41.160568 23005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:41.160851 23005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:41.161994 23005 hybrid_clock.cc:648] HybridClock initialized: now 1786373861161976 us; error 42 us; skew 500 ppm
I20260810 14:57:41.163097 22998 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } attempt: 2
I20260810 14:57:41.163515 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.163381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:38409 (local address 127.0.0.1:48934)
0810 14:57:41.163445 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.163485 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:38409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 14:57:41.163894 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.163793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:38409 (local address 127.0.0.1:48942)
0810 14:57:41.163843 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.163870 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:38409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 14:57:41.163995 22998 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.125:38409: Network error: Client connection negotiation failed: client connection to 127.30.74.125:38409: connect: Connection refused (error 111)
Aug 10 14:57:41 dist-test-slave-0b58 krb5kdc[22910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373861, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:41.165019 23005 init.cc:377] Logged in from keytab as kudu/127.30.74.125@KRBTEST.COM (short username kudu)
I20260810 14:57:41.165465 23005 webserver.cc:460] Webserver started at http://127.30.74.125:33057/ using document root <none> and password file <none>
I20260810 14:57:41.165658 23005 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:41.165699 23005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:41.165793 23005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:41.166874 23005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/data/instance:
uuid: "5f7753a9371744789600549dccaa815a"
format_stamp: "Formatted at 2026-08-10 14:57:41 on dist-test-slave-0b58"
server_key: "49bb2decf8e589f26b8fa8ca1b61aba2"
server_key_iv: "81e52bf9b359e1d235f3d93e7ddb211b"
server_key_version: "encryptionkey@0"
I20260810 14:57:41.167209 23005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/wal/instance:
uuid: "5f7753a9371744789600549dccaa815a"
format_stamp: "Formatted at 2026-08-10 14:57:41 on dist-test-slave-0b58"
server_key: "49bb2decf8e589f26b8fa8ca1b61aba2"
server_key_iv: "81e52bf9b359e1d235f3d93e7ddb211b"
server_key_version: "encryptionkey@0"
I20260810 14:57:41.168609 23005 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 14:57:41.169368 23022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:41.169572 23005 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 14:57:41.169663 23005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/wal
uuid: "5f7753a9371744789600549dccaa815a"
format_stamp: "Formatted at 2026-08-10 14:57:41 on dist-test-slave-0b58"
server_key: "49bb2decf8e589f26b8fa8ca1b61aba2"
server_key_iv: "81e52bf9b359e1d235f3d93e7ddb211b"
server_key_version: "encryptionkey@0"
I20260810 14:57:41.169726 23005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:41.182178 23005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:41.184432 23005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:41.184559 23005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:41.188510 23005 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.125:38409
I20260810 14:57:41.188599 23080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.125:38409 every 8 connection(s)
I20260810 14:57:41.188956 23005 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/data/info.pb
I20260810 14:57:41.189973 23081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:41.191586 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 23005
I20260810 14:57:41.191711 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-1/wal/instance
I20260810 14:57:41.191896 31017 external_mini_cluster.cc:1428] Setting key 639107c6d2cfa3d841a582e0314b8188
I20260810 14:57:41.191686 23081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:57:41 dist-test-slave-0b58 krb5kdc[22910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373861, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.125@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.74.124@KRBTEST.COM; defaulting to no policy
I20260810 14:57:41.202813 23001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.192601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33894 (local address 127.30.74.126:34437)
0810 14:57:41.192653 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.192655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.192988 (+ 333us) server_negotiation.cc:407] Connection header received
0810 14:57:41.193773 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.193775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.193788 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.193846 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.194526 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.195048 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.196109 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.196526 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.196610 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.199157 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:41.199159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:41.199160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:41.199177 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:41.201139 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.201576 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.201577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.201577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.201677 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.201996 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.201997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.201997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.202216 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:41.202281 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.202624 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.202719 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 14:57:41.203013 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.192606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:34437 (local address 127.0.0.1:33894)
0810 14:57:41.192827 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.192887 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:41.193647 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:41.193936 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:41.193941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:41.194210 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:41.194419 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.194423 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.195154 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.195155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.195978 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.195980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.196626 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.196627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.196743 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.197487 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:41.197495 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:41.199039 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:41.201233 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.201244 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.201246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.201496 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.201768 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.201769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.201769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.201895 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.202291 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:41.202297 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:41.202433 (+ 136us) client_negotiation.cc:769] Sending connection context
0810 14:57:41.202527 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 14:57:41.202762 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":106,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.124@KRBTEST.COM" created.
I20260810 14:57:41.203717 23081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:57:41 dist-test-slave-0b58 krb5kdc[22910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373861, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.125@KRBTEST.COM for kudu/127.30.74.125@KRBTEST.COM
I20260810 14:57:41.212196 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.204045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:38409 (local address 127.0.0.1:48958)
0810 14:57:41.204107 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.204119 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:41.204169 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:41.204505 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:41.204506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:41.204673 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:41.204747 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.204748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.205642 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.205644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.206202 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.206204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.206842 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.206843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.206909 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.207418 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:41.207423 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:41.208752 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:41.210969 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.210970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.210970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.211185 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.211484 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.211485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.211487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.211568 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.211933 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:41.211934 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:41.211974 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 14:57:41.212026 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 14:57:41.212122 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 14:57:41.212395 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.204131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48958 (local address 127.30.74.125:38409)
0810 14:57:41.204367 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.204371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.204389 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:57:41.204407 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.204409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.204420 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.204506 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.204835 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.205551 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.206310 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.206755 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.206831 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.208845 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:41.208857 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:41.208858 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:41.208866 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:41.210857 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.211292 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.211296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.211297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.211366 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.211643 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.211644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.211644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.211861 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:41.211958 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.212067 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.212224 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":199,"thread_start_us":167,"threads_started":1}
I20260810 14:57:41.213126 23081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:57:41.213536 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.213406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:42341 (local address 127.0.0.1:41666)
0810 14:57:41.213459 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.213511 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:42341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 14:57:41.213750 23026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.74.124:42341: connect: Connection refused (error 111)
I20260810 14:57:41.214355 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.214265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:42341 (local address 127.0.0.1:41674)
0810 14:57:41.214305 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.214331 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:42341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260810 14:57:41.214479 23081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.124:42341: Network error: Client connection negotiation failed: client connection to 127.30.74.124:42341: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.124.keytab.
Entry for principal kudu/127.30.74.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.124.keytab.
I20260810 14:57:41.223023 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.124:42341
--webserver_interface=127.30.74.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.124.keytab
--principal=kudu/127.30.74.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:44743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:34437,127.30.74.125:38409,127.30.74.124:42341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:57:41.255167 22998 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } attempt: 3
Aug 10 14:57:41 dist-test-slave-0b58 krb5kdc[22910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373860, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for kudu/127.30.74.125@KRBTEST.COM
I20260810 14:57:41.262960 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.255597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:38409 (local address 127.0.0.1:48966)
0810 14:57:41.255672 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.255692 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:41.255768 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:41.255956 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:41.255958 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:41.256132 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:41.256201 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.256202 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.256913 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.256915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.257534 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.257537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.257986 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.257987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.258068 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.258553 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:41.258564 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:41.260070 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:41.261869 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.261870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.261870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.262050 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.262265 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.262266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.262267 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.262340 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.262611 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:41.262613 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:41.262662 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 14:57:41.262742 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 14:57:41.262855 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 14:57:41.262961 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.255680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48966 (local address 127.30.74.125:38409)
0810 14:57:41.255754 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.255757 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.255769 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:57:41.255869 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.255870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.255882 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.255948 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.256310 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.256808 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.257624 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.257874 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.257972 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.260170 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:41.260171 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:41.260171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:41.260179 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:41.261753 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.262127 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.262128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.262129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.262196 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.262407 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.262408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.262409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.262534 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:41.262608 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.262780 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.262867 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 14:57:41.263581 22998 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:57:41.263693 23081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } attempt: 1
I20260810 14:57:41.263969 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.263874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:42341 (local address 127.0.0.1:41686)
0810 14:57:41.263912 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.263941 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:42341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260810 14:57:41.264122 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.263998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:42341 (local address 127.0.0.1:41698)
0810 14:57:41.264064 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.264090 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:42341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 14:57:41.264328 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.264218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:42341 (local address 127.0.0.1:41704)
0810 14:57:41.264277 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.264300 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:42341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260810 14:57:41.264405 22998 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.124:42341: Network error: Client connection negotiation failed: client connection to 127.30.74.124:42341: connect: Connection refused (error 111)
I20260810 14:57:41.264494 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.264409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:42341 (local address 127.0.0.1:41716)
0810 14:57:41.264448 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.264470 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:42341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 14:57:41.264596 23081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.124:42341: Network error: Client connection negotiation failed: client connection to 127.30.74.124:42341: connect: Connection refused (error 111)
I20260810 14:57:41.295576 22998 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } attempt: 1
I20260810 14:57:41.296098 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.295944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:42341 (local address 127.0.0.1:41720)
0810 14:57:41.296019 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.296067 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:42341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 14:57:41.296537 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.296430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:42341 (local address 127.0.0.1:41726)
0810 14:57:41.296483 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.296516 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:42341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 14:57:41.296635 22998 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.124:42341: Network error: Client connection negotiation failed: client connection to 127.30.74.124:42341: connect: Connection refused (error 111)
W20260810 14:57:41.307188 23088 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:57:41.307448 23088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:41.307480 23088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:41.307525 23088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:41.308748 23088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:41.308807 23088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:41.308831 23088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:41.308852 23088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:41.308880 23088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:41.310446 23088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:44743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:34437,127.30.74.125:38409,127.30.74.124:42341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.124.keytab
--principal=kudu/127.30.74.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.124:42341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.74.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:41.310730 23088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:41.310987 23088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:41.316500 23093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:41.316460 23088 server_base.cc:1034] running on GCE node
W20260810 14:57:41.316593 23094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:41.316438 23096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:41.316982 23088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:41.317296 23088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:41.318441 23088 hybrid_clock.cc:648] HybridClock initialized: now 1786373861318417 us; error 44 us; skew 500 ppm
Aug 10 14:57:41 dist-test-slave-0b58 krb5kdc[22910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373861, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:41.320878 23088 init.cc:377] Logged in from keytab as kudu/127.30.74.124@KRBTEST.COM (short username kudu)
I20260810 14:57:41.321386 23088 webserver.cc:460] Webserver started at http://127.30.74.124:38253/ using document root <none> and password file <none>
I20260810 14:57:41.321622 23088 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:41.321691 23088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:41.321851 23088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:41.322947 23088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/data/instance:
uuid: "5356de5cec7c43c0959f0b14f56e3baf"
format_stamp: "Formatted at 2026-08-10 14:57:41 on dist-test-slave-0b58"
server_key: "d3f6d8e9e6f554be93f94f0e90bd41ac"
server_key_iv: "08a4efffbd459546474e6d9a4526c18c"
server_key_version: "encryptionkey@0"
I20260810 14:57:41.323335 23088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/wal/instance:
uuid: "5356de5cec7c43c0959f0b14f56e3baf"
format_stamp: "Formatted at 2026-08-10 14:57:41 on dist-test-slave-0b58"
server_key: "d3f6d8e9e6f554be93f94f0e90bd41ac"
server_key_iv: "08a4efffbd459546474e6d9a4526c18c"
server_key_version: "encryptionkey@0"
I20260810 14:57:41.324797 23088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:57:41.325552 23105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:41.325729 23088 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 14:57:41.325840 23088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/wal
uuid: "5356de5cec7c43c0959f0b14f56e3baf"
format_stamp: "Formatted at 2026-08-10 14:57:41 on dist-test-slave-0b58"
server_key: "d3f6d8e9e6f554be93f94f0e90bd41ac"
server_key_iv: "08a4efffbd459546474e6d9a4526c18c"
server_key_version: "encryptionkey@0"
I20260810 14:57:41.325920 23088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:41.332726 23081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } attempt: 2
I20260810 14:57:41.333174 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.333042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:42341 (local address 127.0.0.1:41740)
0810 14:57:41.333104 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.333143 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:42341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 14:57:41.333560 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.333459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:42341 (local address 127.0.0.1:41752)
0810 14:57:41.333508 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.333539 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:42341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 14:57:41.333653 23081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.124:42341: Network error: Client connection negotiation failed: client connection to 127.30.74.124:42341: connect: Connection refused (error 111)
I20260810 14:57:41.345811 23088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:41.347681 23088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:41.347854 23088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:41.351975 23088 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.124:42341
I20260810 14:57:41.352010 23165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.124:42341 every 8 connection(s)
I20260810 14:57:41.352487 23088 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/data/info.pb
I20260810 14:57:41.353497 23166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:41.355064 23166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:57:41.358896 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 23088
I20260810 14:57:41.359011 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373586526129-31017-0/minicluster-data/master-2/wal/instance
I20260810 14:57:41.359187 31017 external_mini_cluster.cc:1428] Setting key f9dcf2c3ccdf7e94b9d36524ba976b86
Aug 10 14:57:41 dist-test-slave-0b58 krb5kdc[22910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373861, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.124@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:41.364780 23001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.355815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33900 (local address 127.30.74.126:34437)
0810 14:57:41.355932 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.355935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.356142 (+ 207us) server_negotiation.cc:407] Connection header received
0810 14:57:41.356650 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.356651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.356662 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.356728 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.357314 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.357800 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.358827 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.359084 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.359151 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.361577 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:41.361579 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:41.361580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:41.361589 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:41.363245 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.363661 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.363662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.363662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.363728 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.364039 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.364041 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.364041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.364193 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:41.364287 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.364637 (+ 350us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.364712 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 14:57:41.365020 23168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.355767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:34437 (local address 127.0.0.1:33900)
0810 14:57:41.356013 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.356044 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:41.356556 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:41.356784 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:41.356788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:41.357041 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:41.357238 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.357242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.357895 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.357896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.358727 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.358730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.359172 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.359173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.359355 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.359955 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:41.359964 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:41.361485 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:41.363323 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.363331 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.363333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.363577 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.363845 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.363846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.363846 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.363958 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.364326 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:41.364336 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:41.364473 (+ 137us) client_negotiation.cc:769] Sending connection context
0810 14:57:41.364603 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 14:57:41.364764 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":99,"threads_started":1}
Aug 10 14:57:41 dist-test-slave-0b58 krb5kdc[22910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.124@KRBTEST.COM
I20260810 14:57:41.365653 23166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:57:41.368698 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.359617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41756 (local address 127.30.74.124:42341)
0810 14:57:41.359804 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.359811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.359873 (+ 62us) server_negotiation.cc:407] Connection header received
0810 14:57:41.359936 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.359939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.359961 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.360036 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.360795 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.361558 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.362261 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.362743 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.362826 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.365603 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:41.365611 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:41.365611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:41.365618 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:41.367390 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.367777 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.367779 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.367780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.367833 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.368043 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.368044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.368044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.368212 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:41.368407 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.368514 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.368596 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
Aug 10 14:57:41 dist-test-slave-0b58 krb5kdc[22910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373861, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.124@KRBTEST.COM for kudu/127.30.74.125@KRBTEST.COM
I20260810 14:57:41.371757 22998 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } attempt: 2
I20260810 14:57:41.373075 23168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.365954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:38409 (local address 127.0.0.1:48970)
0810 14:57:41.366011 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.366022 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:41.366059 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:41.366425 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:41.366426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:41.366563 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:41.366607 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.366609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.367218 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.367219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.367953 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.367955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.368464 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.368466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.368536 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.368989 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:41.368994 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:41.370230 (+ 1236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:41.371816 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.371817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.371817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.372056 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.372443 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.372443 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.372444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.372511 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.372759 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:41.372760 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:41.372803 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 14:57:41.372887 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 14:57:41.372980 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 14:57:41.373086 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.366254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48970 (local address 127.30.74.125:38409)
0810 14:57:41.366319 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.366321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.366327 (+ 6us) server_negotiation.cc:407] Connection header received
0810 14:57:41.366348 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.366349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.366357 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.366423 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.366686 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.367137 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.368050 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.368387 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.368459 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.370323 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:41.370325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:41.370325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:41.370333 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:41.371704 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.372147 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.372148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.372148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.372212 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.372564 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.372565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.372565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.372681 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:41.372763 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.372913 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.372984 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 14:57:41.373762 23166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:57:41 dist-test-slave-0b58 krb5kdc[22910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
Aug 10 14:57:41 dist-test-slave-0b58 krb5kdc[22910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373860, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for kudu/127.30.74.124@KRBTEST.COM
I20260810 14:57:41.377720 23001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.369488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33902 (local address 127.30.74.126:34437)
0810 14:57:41.369527 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.369529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.369576 (+ 47us) server_negotiation.cc:407] Connection header received
0810 14:57:41.369650 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.369650 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.369660 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.369714 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.370271 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.370741 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.371457 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.371665 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.371743 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.374682 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:41.374683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:41.374683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:41.374690 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:41.376435 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.376843 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.376844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.376845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.376899 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.377101 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.377102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.377103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.377260 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:41.377441 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.377529 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.377611 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 10 14:57:41 dist-test-slave-0b58 krb5kdc[22910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373861, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.124@KRBTEST.COM for kudu/127.30.74.124@KRBTEST.COM
I20260810 14:57:41.379817 23000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.372029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:42341 (local address 127.0.0.1:41762)
0810 14:57:41.372120 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.372130 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:41.372168 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:41.372482 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:41.372484 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:41.372652 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:41.372694 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.372696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.373741 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.373742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.374536 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.374539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.374991 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.374992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.375052 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.375345 (+ 293us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:41.375351 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:41.376962 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:41.378646 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.378647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.378647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.378891 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.379122 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.379123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.379124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.379204 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.379503 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:41.379504 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:41.379551 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 14:57:41.379625 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 14:57:41.379720 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 14:57:41.379863 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.372314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41762 (local address 127.30.74.124:42341)
0810 14:57:41.372375 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.372377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.372384 (+ 7us) server_negotiation.cc:407] Connection header received
0810 14:57:41.372404 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.372405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.372417 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.372475 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.372793 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.373617 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.374605 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.374933 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.374993 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.377029 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:41.377030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:41.377031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:41.377037 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:41.378530 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.378967 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.378968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.378968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.379036 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.379289 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.379290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.379291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.379419 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:41.379499 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.379676 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.379761 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 14:57:41.382608 22998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76: Bootstrap starting.
I20260810 14:57:41.382805 23168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.374036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:42341 (local address 127.0.0.1:41774)
0810 14:57:41.374296 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.374307 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:41.374358 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:41.374532 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:41.374533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:41.374684 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:41.374718 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.374720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.375746 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.375746 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.376182 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.376183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.376621 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.376622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.376669 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.377284 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:41.377289 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:41.379106 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:41.381530 (+ 2424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.381531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.381532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.381771 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.382034 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.382035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.382036 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.382097 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.382388 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:41.382389 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:41.382428 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 14:57:41.382513 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 14:57:41.382618 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244}
I20260810 14:57:41.383033 23171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.374064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41774 (local address 127.30.74.124:42341)
0810 14:57:41.374223 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.374225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.374359 (+ 134us) server_negotiation.cc:407] Connection header received
0810 14:57:41.374419 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.374420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.374430 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.374507 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.374796 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.375493 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.376321 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.376535 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.376620 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.379183 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:41.379184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:41.379185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:41.379190 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:41.381409 (+ 2219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.381867 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.381868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.381868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.381922 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.382170 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.382171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.382171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.382309 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:41.382386 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.382538 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.382697 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260810 14:57:41.383543 22998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:41.383903 22998 log.cc:824] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76: Log is configured to *not* fsync() on all Append() calls
Aug 10 14:57:41 dist-test-slave-0b58 krb5kdc[22910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.125@KRBTEST.COM
I20260810 14:57:41.384999 22998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76: No bootstrap required, opened a new log
I20260810 14:57:41.386206 23166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf: Bootstrap starting.
I20260810 14:57:41.386919 23166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:41.387241 23166 log.cc:824] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:41.387310 22998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } }
I20260810 14:57:41.387466 22998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eade91c12f9499889b5e79c565d4f76, State: Initialized, Role: FOLLOWER
I20260810 14:57:41.387507 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.378655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48982 (local address 127.30.74.125:38409)
0810 14:57:41.378695 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.378697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.378704 (+ 7us) server_negotiation.cc:407] Connection header received
0810 14:57:41.378727 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.378728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.378736 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.378793 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.379374 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.380106 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.381060 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.381403 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.381482 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.384939 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:41.384940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:41.384940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:41.384948 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:41.386357 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.386681 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.386682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.386682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.386739 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.386958 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.386959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.386960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.387093 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:41.387268 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.387314 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.387406 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:57:41.387562 22998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } }
I20260810 14:57:41.387856 23174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } } }
I20260810 14:57:41.387921 22998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:41.387952 23174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:57:41.389160 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 14:57:41.389915 23187 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:57:41.390022 23187 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:57:41.390074 23166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf: No bootstrap required, opened a new log
I20260810 14:57:41.390182 22969 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1eade91c12f9499889b5e79c565d4f76"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33902
I20260810 14:57:41.390304 22969 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 14:57:41.390343 22969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:41.391091 22969 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } }
I20260810 14:57:41.391305 22969 leader_election.cc:288] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [CANDIDATE]: Term 1 election: Requested vote from peers 5f7753a9371744789600549dccaa815a (127.30.74.125:38409), 5356de5cec7c43c0959f0b14f56e3baf (127.30.74.124:42341)
I20260810 14:57:41.392344 23166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } }
I20260810 14:57:41.392488 23166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5356de5cec7c43c0959f0b14f56e3baf, State: Initialized, Role: FOLLOWER
I20260810 14:57:41.392563 23166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } }
I20260810 14:57:41.392856 23166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:41.392930 23135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1eade91c12f9499889b5e79c565d4f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5356de5cec7c43c0959f0b14f56e3baf"
I20260810 14:57:41.393059 23135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:41.390779 22924 master_runner.cc:186] Error getting master registration for 127.30.74.125:38409: OK, instance_id { permanent_uuid: "5f7753a9371744789600549dccaa815a" instance_seqno: 1786373861187344 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:57:41.393784 22924 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 14:57:41.394369 23135 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1eade91c12f9499889b5e79c565d4f76 in term 1.
I20260810 14:57:41.394627 22945 leader_election.cc:302] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1eade91c12f9499889b5e79c565d4f76, 5356de5cec7c43c0959f0b14f56e3baf; no voters:
I20260810 14:57:41.394762 23174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:41.394929 23174 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [term 1 LEADER]: Becoming Leader. State: Replica: 1eade91c12f9499889b5e79c565d4f76, State: Running, Role: LEADER
I20260810 14:57:41.395023 23174 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } }
I20260810 14:57:41.395392 23174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1eade91c12f9499889b5e79c565d4f76. Latest consensus state: current_term: 1 leader_uuid: "1eade91c12f9499889b5e79c565d4f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } } }
I20260810 14:57:41.395498 23174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [sys.catalog]: This master's current role is: LEADER
W20260810 14:57:41.395247 23207 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:57:41.395745 23207 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:57:41.395349 23194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } } }
I20260810 14:57:41.395820 23194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:57:41.396648 23212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:41.396947 23212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:41.398970 23135 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf [term 1 FOLLOWER]: Refusing update from remote peer 1eade91c12f9499889b5e79c565d4f76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:57:41.399343 23174 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:57:41.397549 23088 master_runner.cc:186] Error getting master registration for 127.30.74.125:38409: OK, instance_id { permanent_uuid: "5f7753a9371744789600549dccaa815a" instance_seqno: 1786373861187344 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:57:41.400334 23088 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 14:57:41.400918 23194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1eade91c12f9499889b5e79c565d4f76. Latest consensus state: current_term: 1 leader_uuid: "1eade91c12f9499889b5e79c565d4f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } } }
I20260810 14:57:41.401069 23194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:57:41.403693 23208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1eade91c12f9499889b5e79c565d4f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } } }
I20260810 14:57:41.403820 23208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:41.404543 23194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1eade91c12f9499889b5e79c565d4f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } } }
I20260810 14:57:41.404635 23194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:57:41.404717 23214 mvcc.cc:205] Tried to move back new op lower bound from 7316987336286638080 to 7316987336274616320. Current Snapshot: MvccSnapshot[applied={T|T < 7316987336286638080}]
I20260810 14:57:41.404918 23216 mvcc.cc:205] Tried to move back new op lower bound from 7316987336286638080 to 7316987336274616320. Current Snapshot: MvccSnapshot[applied={T|T < 7316987336286638080}]
I20260810 14:57:41.405376 23001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.392456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33904 (local address 127.30.74.126:34437)
0810 14:57:41.392499 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.392502 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.393126 (+ 624us) server_negotiation.cc:407] Connection header received
0810 14:57:41.393312 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.393314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.393325 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.393365 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.394705 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.395427 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.398545 (+ 3118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.398831 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.398878 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.401710 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:41.401712 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:41.401712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:41.401720 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:41.403695 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.404239 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.404241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.404241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.404329 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.404576 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.404577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.404578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.404720 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:41.404801 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.405200 (+ 399us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.405288 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:57:41.405443 23212 catalog_manager.cc:1347] Generated new cluster ID: 1a2ec70cf62d46e6bf273885f516766e
I20260810 14:57:41.405493 23212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:41.409148 23171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.392964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41776 (local address 127.30.74.124:42341)
0810 14:57:41.393014 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.393015 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.399361 (+ 6346us) server_negotiation.cc:407] Connection header received
0810 14:57:41.399465 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.399467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.399478 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.399510 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.400340 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.401047 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.402089 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.402430 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.402478 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.405032 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:41.405034 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:41.405034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:41.405042 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:41.407414 (+ 2372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.408118 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.408119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.408120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.408186 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.408490 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.408491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.408491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.408612 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:41.408689 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.408980 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.409062 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 14:57:41.412030 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.392513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48994 (local address 127.30.74.125:38409)
0810 14:57:41.392550 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.392552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.394149 (+ 1597us) server_negotiation.cc:407] Connection header received
0810 14:57:41.394256 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.394257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.394270 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.394303 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.395698 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.396428 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.406701 (+ 10273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.407027 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.407075 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.408610 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:41.408612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:41.408612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:41.408620 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:41.410664 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.411131 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.411132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.411133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.411204 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.411420 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.411422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.411422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.411573 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:41.411664 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.411861 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.411941 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 14:57:41.418474 23212 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:41.419224 23212 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:41.424779 23081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } attempt: 3
Aug 10 14:57:41 dist-test-slave-0b58 krb5kdc[22910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373861, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.125@KRBTEST.COM for kudu/127.30.74.124@KRBTEST.COM
I20260810 14:57:41.434243 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.425101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:42341 (local address 127.0.0.1:41782)
0810 14:57:41.425150 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:41.425164 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:41.425210 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:41.425407 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:41.425409 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:41.425610 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:41.425658 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.425660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.426638 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.426639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.427435 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:41.427437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.428772 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.428773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:41.428830 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.429364 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:41.429370 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:41.430899 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:41.433036 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.433037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.433037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.433278 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.433497 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:41.433498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:41.433498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:41.433564 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:41.433882 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:41.433883 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:41.433932 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 14:57:41.434000 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 14:57:41.434124 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 14:57:41.434818 23171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.425229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41782 (local address 127.30.74.124:42341)
0810 14:57:41.425270 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.425272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.425280 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:57:41.425302 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.425303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.425312 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.425361 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.425737 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.426556 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.428354 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.428658 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.428715 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.431006 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:41.431008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:41.431008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:41.431016 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:41.432878 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.433359 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.433360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.433361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.433425 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:41.433635 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:41.433636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:41.433636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:41.433774 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:41.433851 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.434629 (+ 778us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.434716 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:57:41.435182 23212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76: Generated new TSK 0
I20260810 14:57:41.435385 23212 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:57:41.436208 23081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a: Bootstrap starting.
I20260810 14:57:41.437012 23081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:41.437326 23081 log.cc:824] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:41.438263 23081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a: No bootstrap required, opened a new log
I20260810 14:57:41.440335 23081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } }
I20260810 14:57:41.440483 23081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f7753a9371744789600549dccaa815a, State: Initialized, Role: FOLLOWER
I20260810 14:57:41.440572 23081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } }
I20260810 14:57:41.440932 23081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:41.441002 23052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1eade91c12f9499889b5e79c565d4f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5f7753a9371744789600549dccaa815a"
I20260810 14:57:41.441100 23052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:41.442087 23052 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1eade91c12f9499889b5e79c565d4f76 in term 1.
I20260810 14:57:41.442315 23051 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a [term 1 FOLLOWER]: Refusing update from remote peer 1eade91c12f9499889b5e79c565d4f76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:57:41.443233 23174 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:57:41.446918 23237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1eade91c12f9499889b5e79c565d4f76. Latest consensus state: current_term: 1 leader_uuid: "1eade91c12f9499889b5e79c565d4f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } } }
I20260810 14:57:41.447021 23237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:57:41.447077 23237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1eade91c12f9499889b5e79c565d4f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } } }
I20260810 14:57:41.447131 23237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:57:41.440884 23221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } } }
I20260810 14:57:41.450703 23221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:57:41.452077 23233 catalog_manager.cc:1259] Loaded cluster ID: 1a2ec70cf62d46e6bf273885f516766e
I20260810 14:57:41.452162 23233 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a: loading cluster ID for follower catalog manager: success
I20260810 14:57:41.453825 23233 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a: acquiring CA information for follower catalog manager: success
I20260810 14:57:41.454258 23233 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5f7753a9371744789600549dccaa815a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 14:57:41.473857 23085 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:41.471986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48998 (local address 127.30.74.125:38409)
0810 14:57:41.472035 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.472037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.473050 (+ 1013us) server_negotiation.cc:407] Connection header received
0810 14:57:41.473194 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.473195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.473206 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.473238 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.473737 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48998: BlockingRecv error: recv got EOF from 127.0.0.1:48998 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 14:57:41.474905 23171 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:41.472393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41784 (local address 127.30.74.124:42341)
0810 14:57:41.472446 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.472448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.474321 (+ 1873us) server_negotiation.cc:407] Connection header received
0810 14:57:41.474429 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.474430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.474441 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.474472 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.474828 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41784: BlockingRecv error: recv got EOF from 127.0.0.1:41784 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260810 14:57:41.475946 23001 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:41.472694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33914 (local address 127.30.74.126:34437)
0810 14:57:41.472752 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.472754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.475233 (+ 2479us) server_negotiation.cc:407] Connection header received
0810 14:57:41.475417 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.475418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.475430 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.475461 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.475876 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33914: BlockingRecv error: recv got EOF from 127.0.0.1:33914 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 14:57:41.481518 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.477829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49008 (local address 127.30.74.125:38409)
0810 14:57:41.477879 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.477881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.478084 (+ 203us) server_negotiation.cc:407] Connection header received
0810 14:57:41.478197 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.478198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.478245 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.478292 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:41.478504 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.479318 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.480517 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.480829 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.480881 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.481113 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:41.481253 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:41.481306 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.481423 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.481444 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260810 14:57:41.485193 23171 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:41.484459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41794 (local address 127.30.74.124:42341)
0810 14:57:41.484511 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.484512 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.484618 (+ 106us) server_negotiation.cc:407] Connection header received
0810 14:57:41.484733 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.484734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.484753 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.484782 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.485140 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41794: BlockingRecv error: recv got EOF from 127.0.0.1:41794 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 14:57:41.491221 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.487755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49020 (local address 127.30.74.125:38409)
0810 14:57:41.487793 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.487795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.488040 (+ 245us) server_negotiation.cc:407] Connection header received
0810 14:57:41.488133 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.488134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.488144 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.488171 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:41.488403 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.489185 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.490403 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.490702 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.490744 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.490953 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:41.490987 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:41.491015 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.491123 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.491144 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260810 14:57:41.494741 23171 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:41.493949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41808 (local address 127.30.74.124:42341)
0810 14:57:41.493994 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.493996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.494180 (+ 184us) server_negotiation.cc:407] Connection header received
0810 14:57:41.494275 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.494276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.494287 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.494316 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.494680 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41808: BlockingRecv error: recv got EOF from 127.0.0.1:41808 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:57:41.501561 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.498076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49028 (local address 127.30.74.125:38409)
0810 14:57:41.498120 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.498122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.498387 (+ 265us) server_negotiation.cc:407] Connection header received
0810 14:57:41.498490 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.498491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.498501 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.498530 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:41.498719 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.499500 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.500715 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.501025 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.501068 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.501264 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:41.501299 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:41.501337 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.501457 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.501478 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 14:57:41.505088 23171 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:41.504197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41816 (local address 127.30.74.124:42341)
0810 14:57:41.504241 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.504243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.504506 (+ 263us) server_negotiation.cc:407] Connection header received
0810 14:57:41.504607 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.504608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.504620 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.504650 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.505030 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41816: BlockingRecv error: recv got EOF from 127.0.0.1:41816 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:57:41.514194 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.510393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49044 (local address 127.30.74.125:38409)
0810 14:57:41.510437 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.510439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.510751 (+ 312us) server_negotiation.cc:407] Connection header received
0810 14:57:41.510870 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.510871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.510881 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.510911 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:41.511091 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.511884 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.513365 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.513668 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.513709 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.513921 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:41.513956 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:41.513986 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.514092 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.514113 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 14:57:41.517733 23171 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:41.516799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41832 (local address 127.30.74.124:42341)
0810 14:57:41.516857 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.516859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.517137 (+ 278us) server_negotiation.cc:407] Connection header received
0810 14:57:41.517246 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.517247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.517258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.517288 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.517664 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41832: BlockingRecv error: recv got EOF from 127.0.0.1:41832 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:57:41.530930 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.527293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49060 (local address 127.30.74.125:38409)
0810 14:57:41.527342 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.527344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.527645 (+ 301us) server_negotiation.cc:407] Connection header received
0810 14:57:41.527754 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.527755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.527766 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.527796 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:41.528004 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.528812 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.530058 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.530360 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.530402 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.530604 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:41.530639 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:41.530667 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.530809 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.530844 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 14:57:41.534569 23171 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:41.533647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41846 (local address 127.30.74.124:42341)
0810 14:57:41.533692 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.533694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.533976 (+ 282us) server_negotiation.cc:407] Connection header received
0810 14:57:41.534078 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.534079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.534092 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.534123 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.534504 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41846: BlockingRecv error: recv got EOF from 127.0.0.1:41846 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:57:41.555760 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.552099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49070 (local address 127.30.74.125:38409)
0810 14:57:41.552155 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.552157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.552551 (+ 394us) server_negotiation.cc:407] Connection header received
0810 14:57:41.552665 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.552666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.552678 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.552710 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:41.552932 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.553647 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.554845 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.555132 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.555183 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.555468 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:41.555505 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:41.555538 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.555654 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.555677 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260810 14:57:41.559592 23171 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:41.558611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41850 (local address 127.30.74.124:42341)
0810 14:57:41.558655 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.558657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.558961 (+ 304us) server_negotiation.cc:407] Connection header received
0810 14:57:41.559072 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.559073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.559086 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.559119 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.559529 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41850: BlockingRecv error: recv got EOF from 127.0.0.1:41850 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 14:57:41.596707 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.593149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49076 (local address 127.30.74.125:38409)
0810 14:57:41.593200 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.593202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.593526 (+ 324us) server_negotiation.cc:407] Connection header received
0810 14:57:41.593639 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.593640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.593652 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.593682 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:41.593905 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.594644 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.595793 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.596076 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.596124 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.596380 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:41.596420 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:41.596449 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.596601 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.596624 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 14:57:41.600643 23171 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:41.599641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41852 (local address 127.30.74.124:42341)
0810 14:57:41.599696 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.599698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.600023 (+ 325us) server_negotiation.cc:407] Connection header received
0810 14:57:41.600130 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.600131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.600144 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.600174 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.600582 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41852: BlockingRecv error: recv got EOF from 127.0.0.1:41852 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 14:57:41.668635 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.666320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49086 (local address 127.30.74.125:38409)
0810 14:57:41.666443 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.666446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.666496 (+ 50us) server_negotiation.cc:407] Connection header received
0810 14:57:41.666549 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.666550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.666560 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.666613 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:41.666770 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.667249 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.667974 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.668150 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.668200 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.668373 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:41.668399 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:41.668437 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.668504 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.668520 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260810 14:57:41.671052 23171 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:41.670430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41854 (local address 127.30.74.124:42341)
0810 14:57:41.670531 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.670533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.670565 (+ 32us) server_negotiation.cc:407] Connection header received
0810 14:57:41.670617 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.670618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.670628 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.670690 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.670998 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41854: BlockingRecv error: recv got EOF from 127.0.0.1:41854 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 14:57:41.802898 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:41.800319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49098 (local address 127.30.74.125:38409)
0810 14:57:41.800396 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.800398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.800515 (+ 117us) server_negotiation.cc:407] Connection header received
0810 14:57:41.800634 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.800635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.800647 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.800721 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:41.800863 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.801373 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.802183 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:41.802391 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:41.802472 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:41.802626 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:41.802668 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:41.802731 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:41.802788 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 14:57:41.802807 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 14:57:41.805303 23171 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:41.804652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41868 (local address 127.30.74.124:42341)
0810 14:57:41.804728 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:41.804730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:41.804774 (+ 44us) server_negotiation.cc:407] Connection header received
0810 14:57:41.804834 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:41.804835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:41.804845 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:41.804891 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:41.805261 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41868: BlockingRecv error: recv got EOF from 127.0.0.1:41868 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 14:57:41.919186 23208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1eade91c12f9499889b5e79c565d4f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eade91c12f9499889b5e79c565d4f76" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 34437 } } peers { permanent_uuid: "5f7753a9371744789600549dccaa815a" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 38409 } } peers { permanent_uuid: "5356de5cec7c43c0959f0b14f56e3baf" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 42341 } } }
I20260810 14:57:41.919380 23208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eade91c12f9499889b5e79c565d4f76 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:42.065147 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:42.062720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49100 (local address 127.30.74.125:38409)
0810 14:57:42.062801 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:42.062803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:42.062815 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:57:42.062892 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:42.062893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:42.062903 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:42.062966 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:42.063101 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:42.063621 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:42.064505 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:42.064694 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:42.064768 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:42.064943 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:42.064970 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:42.065010 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:42.065070 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 14:57:42.065085 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 14:57:42.067795 23171 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:42.067242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41880 (local address 127.30.74.124:42341)
0810 14:57:42.067305 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:42.067306 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:42.067314 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:57:42.067341 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:42.067342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:42.067351 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:42.067404 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:42.067743 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41880: BlockingRecv error: recv got EOF from 127.0.0.1:41880 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 14:57:42.396282 23207 catalog_manager.cc:1259] Loaded cluster ID: 1a2ec70cf62d46e6bf273885f516766e
I20260810 14:57:42.396364 23207 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf: loading cluster ID for follower catalog manager: success
I20260810 14:57:42.397282 23207 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf: acquiring CA information for follower catalog manager: success
I20260810 14:57:42.397629 23207 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5356de5cec7c43c0959f0b14f56e3baf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:57:42.583823 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:42.581121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49110 (local address 127.30.74.125:38409)
0810 14:57:42.581229 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:42.581231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:42.581416 (+ 185us) server_negotiation.cc:407] Connection header received
0810 14:57:42.581536 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:42.581537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:42.581549 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:42.581622 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:42.581770 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:42.582278 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:42.583079 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:42.583290 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:42.583368 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:42.583527 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:42.583567 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:42.583630 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:42.583693 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 14:57:42.583715 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":56,"threads_started":1}
I20260810 14:57:42.588532 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:42.585823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41882 (local address 127.30.74.124:42341)
0810 14:57:42.585952 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:42.585955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:42.585965 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:57:42.585993 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:42.585993 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:42.586057 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:42.586160 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:42.586274 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:42.586799 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:42.587635 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:42.587920 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:42.587981 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:42.588143 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:42.588286 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:42.588398 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:42.588415 (+ 17us) server_negotiation.cc:299] Negotiation successful
0810 14:57:42.588436 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":56,"threads_started":1}
I20260810 14:57:42.593654 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:42.590327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49122 (local address 127.30.74.125:38409)
0810 14:57:42.590412 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:42.590414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:42.590589 (+ 175us) server_negotiation.cc:407] Connection header received
0810 14:57:42.590682 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:42.590683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:42.590697 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:42.590743 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:42.590912 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:42.591681 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:42.592630 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:42.592961 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:42.593028 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:42.593178 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:42.593214 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:42.593278 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:42.593528 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 14:57:42.593566 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 14:57:42.593900 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:42.590678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41894 (local address 127.30.74.124:42341)
0810 14:57:42.590730 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:42.590732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:42.590964 (+ 232us) server_negotiation.cc:407] Connection header received
0810 14:57:42.591025 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:42.591026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:42.591036 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:42.591087 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:42.591209 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:42.591894 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:42.592903 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:42.593229 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:42.593291 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:42.593590 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:42.593626 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:42.593656 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:42.593771 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 14:57:42.593826 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 14:57:42.593977 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:42.590299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33924 (local address 127.30.74.126:34437)
0810 14:57:42.590480 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:42.590482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:42.590574 (+ 92us) server_negotiation.cc:407] Connection header received
0810 14:57:42.590653 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:42.590654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:42.590715 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:42.590810 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:42.590924 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:42.591562 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:42.592669 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:42.593022 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:42.593066 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:42.593504 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:42.593634 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:42.593680 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:42.593865 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 14:57:42.593892 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":60,"threads_started":1}
I20260810 14:57:42.596345 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 22924
I20260810 14:57:42.604645 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 23005
I20260810 14:57:42.611400 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 23088
2026-08-10T14:57:42Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2951 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:57:42 dist-test-slave-0b58 krb5kdc[23387](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:57:42 dist-test-slave-0b58 krb5kdc[23387](info): set up 2 sockets
Aug 10 14:57:42 dist-test-slave-0b58 krb5kdc[23387](info): commencing operation
krb5kdc: starting...
W20260810 14:57:43.701783 31017 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/test-user.keytab.
Aug 10 14:57:43 dist-test-slave-0b58 krb5kdc[23387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:57:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:57:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.74.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
Entry for principal kudu/127.30.74.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab.
I20260810 14:57:43.831910 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:45157
--webserver_interface=127.30.74.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:43371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:45157,127.30.74.125:34031,127.30.74.124:44539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:57:43.916040 23401 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:57:43.916302 23401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:43.916321 23401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:43.916393 23401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:43.917618 23401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:43.917663 23401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:43.917677 23401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:43.917691 23401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:43.917703 23401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:43.919341 23401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:43371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:45157,127.30.74.125:34031,127.30.74.124:44539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.126.keytab
--principal=kudu/127.30.74.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.126:45157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.74.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:43.919621 23401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:43.919921 23401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 14:57:43.924996 23401 server_base.cc:1034] running on GCE node
W20260810 14:57:43.924974 23406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:43.924961 23407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:43.924978 23409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:43.925571 23401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:43.925897 23401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:43.927068 23401 hybrid_clock.cc:648] HybridClock initialized: now 1786373863927046 us; error 54 us; skew 500 ppm
Aug 10 14:57:43 dist-test-slave-0b58 krb5kdc[23387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373863, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:43.935211 23401 init.cc:377] Logged in from keytab as kudu/127.30.74.126@KRBTEST.COM (short username kudu)
I20260810 14:57:43.935771 23401 webserver.cc:460] Webserver started at http://127.30.74.126:46767/ using document root <none> and password file <none>
I20260810 14:57:43.936020 23401 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:43.936094 23401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:43.936250 23401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:43.937446 23401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/data/instance:
uuid: "b3ab960de4754f24a1dffcfa67427f53"
format_stamp: "Formatted at 2026-08-10 14:57:43 on dist-test-slave-0b58"
server_key: "b0eb45dfd9390dee56e89d1b3b210bf8"
server_key_iv: "31a0c6232d205abfe308284c26fb339a"
server_key_version: "encryptionkey@0"
I20260810 14:57:43.937850 23401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/wal/instance:
uuid: "b3ab960de4754f24a1dffcfa67427f53"
format_stamp: "Formatted at 2026-08-10 14:57:43 on dist-test-slave-0b58"
server_key: "b0eb45dfd9390dee56e89d1b3b210bf8"
server_key_iv: "31a0c6232d205abfe308284c26fb339a"
server_key_version: "encryptionkey@0"
I20260810 14:57:43.939292 23401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:57:43.940083 23416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:43.940313 23401 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:43.940428 23401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/wal
uuid: "b3ab960de4754f24a1dffcfa67427f53"
format_stamp: "Formatted at 2026-08-10 14:57:43 on dist-test-slave-0b58"
server_key: "b0eb45dfd9390dee56e89d1b3b210bf8"
server_key_iv: "31a0c6232d205abfe308284c26fb339a"
server_key_version: "encryptionkey@0"
I20260810 14:57:43.940512 23401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:43.968668 23401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:43.970913 23401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:43.971094 23401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:43.975564 23401 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.126:45157
I20260810 14:57:43.975605 23471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.126:45157 every 8 connection(s)
I20260810 14:57:43.976001 23401 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/data/info.pb
I20260810 14:57:43.976954 23472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:43.978185 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 23401
I20260810 14:57:43.978274 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-0/wal/instance
I20260810 14:57:43.978493 31017 external_mini_cluster.cc:1428] Setting key 9ac16ff5f31327c47cc2b731110b21d2
I20260810 14:57:43.978693 23472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:57:43 dist-test-slave-0b58 krb5kdc[23387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373863, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.74.125@KRBTEST.COM; defaulting to no policy
I20260810 14:57:43.989208 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:43.979424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45157 (local address 127.0.0.1:43172)
0810 14:57:43.979617 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:43.979662 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:43.980144 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:43.980472 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:43.980475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:43.980761 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:43.980923 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:43.980928 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:43.981976 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:43.981978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:43.982642 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:43.982644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:43.983183 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:43.983184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:43.983249 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:43.983918 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:43.983926 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:43.985695 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:43.987550 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:43.987556 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:43.987558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:43.987750 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:43.988059 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:43.988060 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:43.988060 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:43.988149 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:43.988570 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:43.988575 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:43.988638 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 14:57:43.988750 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 14:57:43.988966 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":2,"thread_start_us":115,"threads_started":1}
I20260810 14:57:43.989208 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:43.979455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43172 (local address 127.30.74.126:45157)
0810 14:57:43.979688 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:43.979692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:43.979745 (+ 53us) server_negotiation.cc:407] Connection header received
0810 14:57:43.980337 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:43.980340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:43.980360 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:43.980467 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:43.981029 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:43.981861 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:43.982743 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:43.983063 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:43.983194 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:43.985788 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:43.985795 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:43.985795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:43.985803 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:43.987457 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:43.987857 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:43.987862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:43.987862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:43.987984 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:43.988223 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:43.988224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:43.988224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:43.988495 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:43.988590 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:43.988824 (+ 234us) server_negotiation.cc:299] Negotiation successful
0810 14:57:43.989013 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":108,"server-negotiator.queue_time_us":209,"thread_start_us":95,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.125@KRBTEST.COM" created.
I20260810 14:57:43.990165 23472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:57:43.990551 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:43.990413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:34031 (local address 127.0.0.1:46592)
0810 14:57:43.990464 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:43.990515 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:34031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 14:57:43.990729 23420 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.74.125:34031: connect: Connection refused (error 111)
I20260810 14:57:43.991221 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:43.991108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:34031 (local address 127.0.0.1:46606)
0810 14:57:43.991158 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:43.991190 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:34031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 14:57:43.991387 23472 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.125:34031: Network error: Client connection negotiation failed: client connection to 127.30.74.125:34031: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.125.keytab.
Entry for principal kudu/127.30.74.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.125.keytab.
I20260810 14:57:44.007915 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.125:34031
--webserver_interface=127.30.74.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.125.keytab
--principal=kudu/127.30.74.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:43371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:45157,127.30.74.125:34031,127.30.74.124:44539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:57:44.040634 23472 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } attempt: 1
I20260810 14:57:44.041215 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.041040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:34031 (local address 127.0.0.1:46608)
0810 14:57:44.041117 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.041174 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:34031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 14:57:44.041616 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.041506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:34031 (local address 127.0.0.1:46618)
0810 14:57:44.041555 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.041587 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:34031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 14:57:44.041754 23472 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.125:34031: Network error: Client connection negotiation failed: client connection to 127.30.74.125:34031: connect: Connection refused (error 111)
W20260810 14:57:44.092794 23479 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:57:44.093017 23479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:44.093034 23479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:44.093057 23479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:44.094238 23479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:44.094285 23479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:44.094300 23479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:44.094312 23479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:44.094363 23479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:44.095924 23479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:43371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:45157,127.30.74.125:34031,127.30.74.124:44539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.125.keytab
--principal=kudu/127.30.74.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.125:34031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.74.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:44.096200 23479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:44.096483 23479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:44.101629 23485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:44.101643 23486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:44.101626 23488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:44.101838 23479 server_base.cc:1034] running on GCE node
I20260810 14:57:44.102238 23479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:44.102551 23479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:44.103725 23479 hybrid_clock.cc:648] HybridClock initialized: now 1786373864103727 us; error 64 us; skew 500 ppm
Aug 10 14:57:44 dist-test-slave-0b58 krb5kdc[23387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373864, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:44.106509 23479 init.cc:377] Logged in from keytab as kudu/127.30.74.125@KRBTEST.COM (short username kudu)
I20260810 14:57:44.106980 23479 webserver.cc:460] Webserver started at http://127.30.74.125:45921/ using document root <none> and password file <none>
I20260810 14:57:44.107230 23479 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:44.107304 23479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:44.107457 23479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:44.108671 23479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/data/instance:
uuid: "4c4aba9989f34481a2ac08c88e8a2759"
format_stamp: "Formatted at 2026-08-10 14:57:44 on dist-test-slave-0b58"
server_key: "ffaed920f6cc7b0266ed53ffb4f6f542"
server_key_iv: "6ef685a048043579c9c131a56520f76a"
server_key_version: "encryptionkey@0"
I20260810 14:57:44.109058 23479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/wal/instance:
uuid: "4c4aba9989f34481a2ac08c88e8a2759"
format_stamp: "Formatted at 2026-08-10 14:57:44 on dist-test-slave-0b58"
server_key: "ffaed920f6cc7b0266ed53ffb4f6f542"
server_key_iv: "6ef685a048043579c9c131a56520f76a"
server_key_version: "encryptionkey@0"
I20260810 14:57:44.109939 23472 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } attempt: 2
I20260810 14:57:44.110380 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.110246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:34031 (local address 127.0.0.1:46620)
0810 14:57:44.110315 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.110351 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:34031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 14:57:44.110526 23479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 14:57:44.110780 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.110686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:34031 (local address 127.0.0.1:46634)
0810 14:57:44.110735 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.110761 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.125:34031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 14:57:44.110893 23472 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.125:34031: Network error: Client connection negotiation failed: client connection to 127.30.74.125:34031: connect: Connection refused (error 111)
I20260810 14:57:44.111475 23495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:44.111646 23479 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:44.111742 23479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/wal
uuid: "4c4aba9989f34481a2ac08c88e8a2759"
format_stamp: "Formatted at 2026-08-10 14:57:44 on dist-test-slave-0b58"
server_key: "ffaed920f6cc7b0266ed53ffb4f6f542"
server_key_iv: "6ef685a048043579c9c131a56520f76a"
server_key_version: "encryptionkey@0"
I20260810 14:57:44.111809 23479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:44.130725 23479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:44.133298 23479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:44.133488 23479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:44.137735 23479 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.125:34031
I20260810 14:57:44.137880 23554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.125:34031 every 8 connection(s)
I20260810 14:57:44.138216 23479 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/data/info.pb
I20260810 14:57:44.139201 23555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:44.140902 23555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:57:44.144153 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 23479
I20260810 14:57:44.144308 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-1/wal/instance
I20260810 14:57:44.144510 31017 external_mini_cluster.cc:1428] Setting key d584f30adce651284cc779d59edcdf68
Aug 10 14:57:44 dist-test-slave-0b58 krb5kdc[23387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373864, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.125@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:44.151587 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.141644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43182 (local address 127.30.74.126:45157)
0810 14:57:44.141742 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.141744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.142028 (+ 284us) server_negotiation.cc:407] Connection header received
0810 14:57:44.142471 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.142473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.142484 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.142547 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.143263 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.143839 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.145363 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.145574 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.145636 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.148208 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:44.148210 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:44.148210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:44.148219 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:44.150120 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.150545 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.150547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.150548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.150618 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.150866 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.150868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.150868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.151042 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:44.151124 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:44.151378 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 14:57:44.151471 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:57:44.151726 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.141598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45157 (local address 127.0.0.1:43182)
0810 14:57:44.141861 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.141905 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:44.142377 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:44.142628 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:44.142631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:44.142915 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:44.143173 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:44.143179 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.143932 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.143933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:44.145268 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:44.145271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.145681 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.145682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:44.145843 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.146437 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:44.146447 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:44.148102 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:44.150210 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:44.150217 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:44.150219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:44.150462 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:44.150692 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:44.150692 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:44.150693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:44.150796 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:44.151111 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:44.151116 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:44.151240 (+ 124us) client_negotiation.cc:769] Sending connection context
0810 14:57:44.151324 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 14:57:44.151458 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":171,"threads_started":1}
I20260810 14:57:44.152434 23555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.74.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.74.124@KRBTEST.COM" created.
Aug 10 14:57:44 dist-test-slave-0b58 krb5kdc[23387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373864, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.125@KRBTEST.COM for kudu/127.30.74.125@KRBTEST.COM
I20260810 14:57:44.160859 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.152828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46646 (local address 127.30.74.125:34031)
0810 14:57:44.153024 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.153028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.153045 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:57:44.153062 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.153064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.153075 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.153270 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.153556 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.154295 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.155109 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.155435 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.155522 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.157554 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:44.157566 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:44.157566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:44.157575 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:44.159410 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.159892 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.159896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.159896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.159961 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.160188 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.160189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.160189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.160433 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:44.160507 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:44.160674 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 14:57:44.160759 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":113,"threads_started":1}
I20260810 14:57:44.161006 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.152745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:34031 (local address 127.0.0.1:46646)
0810 14:57:44.152809 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.152818 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:44.152860 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:44.153272 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:44.153273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:44.153420 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:44.153473 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:44.153474 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.154411 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.154412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:44.155011 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:44.155013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.155525 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.155526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:44.155594 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.156062 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:44.156067 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:44.157459 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:44.159513 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:44.159514 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:44.159515 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:44.159777 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:44.160046 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:44.160047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:44.160047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:44.160120 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:44.160506 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:44.160507 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:44.160563 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 14:57:44.160646 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 14:57:44.160831 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":64}
I20260810 14:57:44.161794 23555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:57:44.162257 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.162099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:44539 (local address 127.0.0.1:53782)
0810 14:57:44.162159 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.162220 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 14:57:44.162423 23496 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.74.124:44539: connect: Connection refused (error 111)
I20260810 14:57:44.162906 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.162780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:44539 (local address 127.0.0.1:53792)
0810 14:57:44.162837 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.162872 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 14:57:44.163045 23555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.124:44539: Network error: Client connection negotiation failed: client connection to 127.30.74.124:44539: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.74.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.124.keytab.
Entry for principal kudu/127.30.74.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.124.keytab.
I20260810 14:57:44.175261 31017 external_mini_cluster.cc:1326] Running /tmp/dist-test-task2SWuiv/build/release/bin/kudu
/tmp/dist-test-task2SWuiv/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.124:44539
--webserver_interface=127.30.74.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.124.keytab
--principal=kudu/127.30.74.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.74.84:43371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.74.126:45157,127.30.74.125:34031,127.30.74.124:44539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:57:44.202097 23472 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } attempt: 3
Aug 10 14:57:44 dist-test-slave-0b58 krb5kdc[23387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373863, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for kudu/127.30.74.125@KRBTEST.COM
I20260810 14:57:44.209777 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.202639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46662 (local address 127.30.74.125:34031)
0810 14:57:44.202712 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.202715 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.202726 (+ 11us) server_negotiation.cc:407] Connection header received
0810 14:57:44.202759 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.202760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.202770 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.202870 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.203211 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.203714 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.204532 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.204763 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.204853 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.207094 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:44.207096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:44.207096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:44.207104 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:44.208620 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.209031 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.209031 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.209032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.209088 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.209297 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.209297 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.209297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.209412 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:44.209461 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:44.209632 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 14:57:44.209704 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 14:57:44.209827 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.202542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:34031 (local address 127.0.0.1:46662)
0810 14:57:44.202612 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.202627 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:44.202698 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:44.202878 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:44.202879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:44.203057 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:44.203124 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:44.203126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.203830 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.203832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:44.204433 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:44.204436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.204844 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.204845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:44.204929 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.205436 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:44.205441 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:44.206985 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:44.208712 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:44.208714 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:44.208714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:44.208953 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:44.209158 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:44.209159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:44.209159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:44.209234 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:44.209475 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:44.209476 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:44.209535 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 14:57:44.209616 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 14:57:44.209725 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 14:57:44.210327 23472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:57:44.210729 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.210598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:44539 (local address 127.0.0.1:53794)
0810 14:57:44.210662 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.210701 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 14:57:44.211417 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.211307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:44539 (local address 127.0.0.1:53810)
0810 14:57:44.211363 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.211395 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 14:57:44.211580 23472 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.124:44539: Network error: Client connection negotiation failed: client connection to 127.30.74.124:44539: connect: Connection refused (error 111)
I20260810 14:57:44.212314 23555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } attempt: 1
I20260810 14:57:44.212735 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.212577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:44539 (local address 127.0.0.1:53822)
0810 14:57:44.212660 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.212700 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 14:57:44.213097 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.212987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:44539 (local address 127.0.0.1:53824)
0810 14:57:44.213040 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.213074 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 14:57:44.213225 23555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.124:44539: Network error: Client connection negotiation failed: client connection to 127.30.74.124:44539: connect: Connection refused (error 111)
I20260810 14:57:44.242714 23472 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } attempt: 1
I20260810 14:57:44.243280 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.243117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:44539 (local address 127.0.0.1:53830)
0810 14:57:44.243186 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.243243 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 14:57:44.243718 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.243596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:44539 (local address 127.0.0.1:53834)
0810 14:57:44.243651 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.243685 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 14:57:44.243825 23472 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.124:44539: Network error: Client connection negotiation failed: client connection to 127.30.74.124:44539: connect: Connection refused (error 111)
W20260810 14:57:44.268379 23563 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:57:44.268707 23563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:57:44.268775 23563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:57:44.268826 23563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:57:44.270452 23563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:57:44.270512 23563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:57:44.270526 23563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:57:44.270539 23563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:57:44.270552 23563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:57:44.272248 23563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.74.84:43371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.74.126:45157,127.30.74.125:34031,127.30.74.124:44539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/krb5kdc/kudu_127.30.74.124.keytab
--principal=kudu/127.30.74.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.74.124:44539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.74.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 13:36:56 UTC on ccycloud.quasar-zxzunu.root.comops.site
build id 4577778
I20260810 14:57:44.272590 23563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:57:44.272887 23563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:57:44.278028 23568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:57:44.278033 23569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:44.278121 23563 server_base.cc:1034] running on GCE node
W20260810 14:57:44.278088 23571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:57:44.278573 23563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:57:44.278903 23563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:57:44.280094 23563 hybrid_clock.cc:648] HybridClock initialized: now 1786373864280087 us; error 36 us; skew 500 ppm
I20260810 14:57:44.281380 23555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } attempt: 2
I20260810 14:57:44.281806 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.281649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:44539 (local address 127.0.0.1:53836)
0810 14:57:44.281726 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.281769 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 14:57:44.282191 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.282075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:44539 (local address 127.0.0.1:53846)
0810 14:57:44.282127 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.282158 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.74.124:44539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 14:57:44.282296 23555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.74.124:44539: Network error: Client connection negotiation failed: client connection to 127.30.74.124:44539: connect: Connection refused (error 111)
Aug 10 14:57:44 dist-test-slave-0b58 krb5kdc[23387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373864, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:57:44.283123 23563 init.cc:377] Logged in from keytab as kudu/127.30.74.124@KRBTEST.COM (short username kudu)
I20260810 14:57:44.283558 23563 webserver.cc:460] Webserver started at http://127.30.74.124:45625/ using document root <none> and password file <none>
I20260810 14:57:44.283825 23563 fs_manager.cc:359] Metadata directory not provided
I20260810 14:57:44.283874 23563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:57:44.284026 23563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:57:44.285246 23563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/data/instance:
uuid: "b5a8b09cc9124c9c904306226f6c2b62"
format_stamp: "Formatted at 2026-08-10 14:57:44 on dist-test-slave-0b58"
server_key: "b8ff58b436f5153b9e16cc63bc673eac"
server_key_iv: "7791a8cfeac07d6e6669becb41f614ac"
server_key_version: "encryptionkey@0"
I20260810 14:57:44.285630 23563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/wal/instance:
uuid: "b5a8b09cc9124c9c904306226f6c2b62"
format_stamp: "Formatted at 2026-08-10 14:57:44 on dist-test-slave-0b58"
server_key: "b8ff58b436f5153b9e16cc63bc673eac"
server_key_iv: "7791a8cfeac07d6e6669becb41f614ac"
server_key_version: "encryptionkey@0"
I20260810 14:57:44.287109 23563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 14:57:44.288033 23580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:57:44.288232 23563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 14:57:44.288394 23563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/data,/tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/wal
uuid: "b5a8b09cc9124c9c904306226f6c2b62"
format_stamp: "Formatted at 2026-08-10 14:57:44 on dist-test-slave-0b58"
server_key: "b8ff58b436f5153b9e16cc63bc673eac"
server_key_iv: "7791a8cfeac07d6e6669becb41f614ac"
server_key_version: "encryptionkey@0"
I20260810 14:57:44.288487 23563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:57:44.306636 23563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:57:44.309175 23563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:57:44.309365 23563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:57:44.313745 23563 rpc_server.cc:307] RPC server started. Bound to: 127.30.74.124:44539
I20260810 14:57:44.313782 23641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.74.124:44539 every 8 connection(s)
I20260810 14:57:44.314234 23563 server_base.cc:1166] Dumped server information to /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/data/info.pb
I20260810 14:57:44.315399 23642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:57:44.317173 23642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:57:44.318955 23472 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } attempt: 2
I20260810 14:57:44.322001 31017 external_mini_cluster.cc:1388] Started /tmp/dist-test-task2SWuiv/build/release/bin/kudu as pid 23563
I20260810 14:57:44.322165 31017 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task2SWuiv/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373586526129-31017-0/minicluster-data/master-2/wal/instance
I20260810 14:57:44.322397 31017 external_mini_cluster.cc:1428] Setting key 92d5729e1cdf3f11b43ce649964d1486
Aug 10 14:57:44 dist-test-slave-0b58 krb5kdc[23387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373863, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.126@KRBTEST.COM for kudu/127.30.74.124@KRBTEST.COM
Aug 10 14:57:44 dist-test-slave-0b58 krb5kdc[23387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373864, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.124@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
Aug 10 14:57:44 dist-test-slave-0b58 krb5kdc[23387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.124@KRBTEST.COM
I20260810 14:57:44.329102 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.317904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43184 (local address 127.30.74.126:45157)
0810 14:57:44.318042 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.318045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.318244 (+ 199us) server_negotiation.cc:407] Connection header received
0810 14:57:44.318845 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.318847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.318861 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.318916 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.319941 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.320726 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.322085 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.322438 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.322537 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.325722 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:44.325724 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:44.325725 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:44.325732 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:44.327376 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.327838 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.327839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.327840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.327922 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.328173 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.328174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.328174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.328413 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:44.328496 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:44.328763 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 14:57:44.328930 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 14:57:44.329253 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.319230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:44539 (local address 127.0.0.1:53852)
0810 14:57:44.319303 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.319316 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:44.319368 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:44.319619 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:44.319620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:44.319799 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:44.319845 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:44.319846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.321182 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.321183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:44.321973 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:44.321975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.322716 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.322717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:44.322767 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.323446 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:44.323451 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:44.325065 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:44.327361 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:44.327362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:44.327363 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:44.327621 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:44.327920 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:44.327921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:44.327921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:44.327981 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:44.328704 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:44.328705 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:44.328752 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 14:57:44.328833 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 14:57:44.328997 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":55}
I20260810 14:57:44.329299 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.319318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53852 (local address 127.30.74.124:44539)
0810 14:57:44.319463 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.319467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.319483 (+ 16us) server_negotiation.cc:407] Connection header received
0810 14:57:44.319503 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.319504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.319523 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.319589 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.319947 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.321075 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.322099 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.322605 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.322734 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.325165 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:44.325174 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:44.325174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:44.325182 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:44.327252 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.327720 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.327723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.327724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.327829 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.328339 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.328340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.328340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.328617 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:44.328710 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:44.328891 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 14:57:44.329049 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260810 14:57:44.329420 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.317825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.126:45157 (local address 127.0.0.1:43184)
0810 14:57:44.318093 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.318144 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:44.318712 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:44.319010 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:44.319013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:44.319502 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:44.319838 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:44.319846 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.320866 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.320867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:44.321973 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:44.321976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.322640 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.322641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:44.322744 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.323353 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:44.323363 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:44.325593 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:44.327477 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:44.327483 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:44.327485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:44.327757 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:44.327989 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:44.327990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:44.327990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:44.328103 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:44.328495 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:44.328499 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:44.328607 (+ 108us) client_negotiation.cc:769] Sending connection context
0810 14:57:44.329208 (+ 601us) client_negotiation.cc:240] Negotiation successful
0810 14:57:44.329320 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":116,"threads_started":1}
I20260810 14:57:44.330055 23642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:57:44.331386 23472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53: Bootstrap starting.
I20260810 14:57:44.332231 23472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:44.332621 23472 log.cc:824] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:44.333567 23472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53: No bootstrap required, opened a new log
Aug 10 14:57:44 dist-test-slave-0b58 krb5kdc[23387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373864, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.124@KRBTEST.COM for kudu/127.30.74.125@KRBTEST.COM
I20260810 14:57:44.335656 23647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.322870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53864 (local address 127.30.74.124:44539)
0810 14:57:44.322999 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.323002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.323121 (+ 119us) server_negotiation.cc:407] Connection header received
0810 14:57:44.323199 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.323200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.323211 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.323274 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.323896 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.324658 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.326070 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.326402 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.326471 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.330239 (+ 3768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:44.330240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:44.330241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:44.330248 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:44.333446 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.334126 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.334127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.334127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.334207 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.334535 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.334536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.334536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.334680 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:44.334833 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:44.335111 (+ 278us) server_negotiation.cc:299] Negotiation successful
0810 14:57:44.335508 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260810 14:57:44.336011 23472 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } }
I20260810 14:57:44.336177 23472 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ab960de4754f24a1dffcfa67427f53, State: Initialized, Role: FOLLOWER
I20260810 14:57:44.336292 23472 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } }
I20260810 14:57:44.336625 23649 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } } }
I20260810 14:57:44.336711 23472 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:44.337682 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.330428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46676 (local address 127.30.74.125:34031)
0810 14:57:44.330472 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.330474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.330482 (+ 8us) server_negotiation.cc:407] Connection header received
0810 14:57:44.330510 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.330510 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.330520 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.330577 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.330912 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.331388 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.332080 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.332423 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.332511 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.334720 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:44.334722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:44.334722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:44.334730 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:44.336356 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.336771 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.336772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.336773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.336864 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.337083 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.337084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.337085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.337237 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:44.337319 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:44.337499 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 14:57:44.337585 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 14:57:44.337704 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.330366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.125:34031 (local address 127.0.0.1:46676)
0810 14:57:44.330410 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.330421 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:44.330457 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:44.330581 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:44.330582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:44.330742 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:44.330801 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:44.330803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.331478 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.331479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:44.332000 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:44.332004 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.332510 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.332511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:44.332571 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.333044 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:44.333051 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:44.334631 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:44.336446 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:44.336447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:44.336448 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:44.336668 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:44.336950 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:44.336951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:44.336951 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:44.337019 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:44.337330 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:44.337331 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:44.337384 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 14:57:44.337473 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 14:57:44.337589 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 14:57:44.338068 23642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:57:44.338594 23649 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 14:57:44.339174 23660 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:57:44.339226 23660 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 10 14:57:44 dist-test-slave-0b58 krb5kdc[23387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.126@KRBTEST.COM
I20260810 14:57:44.339928 23401 master_runner.cc:186] Error getting master registration for 127.30.74.125:34031: OK, instance_id { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" instance_seqno: 1786373864136484 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:57:44.343058 23401 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 10 14:57:44 dist-test-slave-0b58 krb5kdc[23387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373864, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.124@KRBTEST.COM for kudu/127.30.74.124@KRBTEST.COM
I20260810 14:57:44.346881 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.338301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:44539 (local address 127.0.0.1:53866)
0810 14:57:44.338350 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.338361 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:44.338409 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:44.338690 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:44.338691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:44.338845 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:44.338884 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:44.338886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.339716 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.339716 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:44.340467 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:44.340470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.341384 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.341384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:44.341431 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.341828 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:44.341833 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:44.343697 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:44.345671 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:44.345672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:44.345673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:44.345898 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:44.346141 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:44.346148 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:44.346149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:44.346221 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:44.346575 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:44.346576 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:44.346614 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 14:57:44.346679 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 14:57:44.346774 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 14:57:44.347260 23647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.338483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53866 (local address 127.30.74.124:44539)
0810 14:57:44.338548 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.338550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.338557 (+ 7us) server_negotiation.cc:407] Connection header received
0810 14:57:44.338577 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.338578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.338589 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.338654 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.338968 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.339636 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.340632 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.341257 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.341323 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.343771 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:44.343772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:44.343772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:44.343778 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:44.345574 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.345987 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.345989 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.345989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.346050 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.346314 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.346315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.346315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.346520 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:44.346583 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:44.347103 (+ 520us) server_negotiation.cc:299] Negotiation successful
0810 14:57:44.347180 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 14:57:44.347491 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.336770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43192 (local address 127.30.74.126:45157)
0810 14:57:44.336811 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.336813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.336888 (+ 75us) server_negotiation.cc:407] Connection header received
0810 14:57:44.336926 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.336927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.336937 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.336988 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.337609 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.338321 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.340127 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.340441 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.340487 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.344319 (+ 3832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:44.344320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:44.344321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:44.344329 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:44.346118 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.346534 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.346535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.346535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.346599 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.346820 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.346821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.346821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.346960 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:44.347102 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:44.347341 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 14:57:44.347434 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 14:57:44.349231 23642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62: Bootstrap starting.
I20260810 14:57:44.349998 23642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:44.350307 23642 log.cc:824] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:44.351125 23642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62: No bootstrap required, opened a new log
Aug 10 14:57:44 dist-test-slave-0b58 krb5kdc[23387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.74.125@KRBTEST.COM
I20260810 14:57:44.353366 23642 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } }
I20260810 14:57:44.353531 23642 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a8b09cc9124c9c904306226f6c2b62, State: Initialized, Role: FOLLOWER
I20260810 14:57:44.353619 23642 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } }
I20260810 14:57:44.353950 23642 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:44.353937 23663 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } } }
I20260810 14:57:44.354317 23663 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:57:44.356222 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.348128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46686 (local address 127.30.74.125:34031)
0810 14:57:44.348191 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.348193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.348202 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:57:44.348322 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.348323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.348340 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.348397 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.349000 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.349543 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.350483 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.350679 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.350764 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.353139 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:44.353141 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:44.353141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:44.353149 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:44.354969 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.355409 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.355411 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.355411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.355479 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.355703 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.355704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.355705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.355845 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:44.356005 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:44.356043 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 14:57:44.356119 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260810 14:57:44.356961 23675 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:57:44.357030 23675 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:57:44.357862 31017 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:57:44.358784 23444 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b3ab960de4754f24a1dffcfa67427f53"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43192
I20260810 14:57:44.358937 23444 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 14:57:44.358990 23444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:44.357575 23563 master_runner.cc:186] Error getting master registration for 127.30.74.125:34031: OK, instance_id { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" instance_seqno: 1786373864136484 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:57:44.360001 23563 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 14:57:44.360023 23444 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } }
I20260810 14:57:44.360494 23444 leader_election.cc:288] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [CANDIDATE]: Term 1 election: Requested vote from peers 4c4aba9989f34481a2ac08c88e8a2759 (127.30.74.125:34031), b5a8b09cc9124c9c904306226f6c2b62 (127.30.74.124:44539)
I20260810 14:57:44.360847 23614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b3ab960de4754f24a1dffcfa67427f53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b5a8b09cc9124c9c904306226f6c2b62"
I20260810 14:57:44.360955 23614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:44.361861 23614 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ab960de4754f24a1dffcfa67427f53 in term 1.
I20260810 14:57:44.362102 23417 leader_election.cc:302] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3ab960de4754f24a1dffcfa67427f53, b5a8b09cc9124c9c904306226f6c2b62; no voters:
I20260810 14:57:44.362257 23649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:57:44.362464 23649 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [term 1 LEADER]: Becoming Leader. State: Replica: b3ab960de4754f24a1dffcfa67427f53, State: Running, Role: LEADER
I20260810 14:57:44.362572 23649 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } }
I20260810 14:57:44.363215 23649 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3ab960de4754f24a1dffcfa67427f53. Latest consensus state: current_term: 1 leader_uuid: "b3ab960de4754f24a1dffcfa67427f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } } }
I20260810 14:57:44.363308 23649 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:44.363515 23683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:57:44.363747 23683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:57:44.365414 23614 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62 [term 1 FOLLOWER]: Refusing update from remote peer b3ab960de4754f24a1dffcfa67427f53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:57:44.365859 23649 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:57:44.367190 23663 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3ab960de4754f24a1dffcfa67427f53. Latest consensus state: current_term: 1 leader_uuid: "b3ab960de4754f24a1dffcfa67427f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } } }
I20260810 14:57:44.367286 23663 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:57:44.373435 23555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } attempt: 3
Aug 10 14:57:44 dist-test-slave-0b58 krb5kdc[23387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373864, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.74.125@KRBTEST.COM for kudu/127.30.74.124@KRBTEST.COM
I20260810 14:57:44.384400 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.373712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.74.124:44539 (local address 127.0.0.1:53892)
0810 14:57:44.373766 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 14:57:44.373776 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 14:57:44.373822 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:57:44.374003 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:57:44.374004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:57:44.374191 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:57:44.374246 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:44.374248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.375701 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.375702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:44.376563 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:57:44.376565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.377041 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.377042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:57:44.377102 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.377611 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:57:44.377617 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:57:44.379318 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:57:44.383119 (+ 3801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:44.383121 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:44.383121 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:44.383369 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:44.383594 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:57:44.383595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:57:44.383595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:57:44.383659 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:57:44.383988 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:57:44.383989 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:57:44.384035 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 14:57:44.384114 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 14:57:44.384276 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 14:57:44.384841 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.373757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53892 (local address 127.30.74.124:44539)
0810 14:57:44.373794 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.373796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.373806 (+ 10us) server_negotiation.cc:407] Connection header received
0810 14:57:44.373884 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.373885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.373895 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.373946 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.374337 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.375110 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.376648 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.376957 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.377042 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.379406 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:44.379407 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:44.379408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:44.379416 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:44.382234 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.383463 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.383464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.383465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.383522 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.383737 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.383738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.383739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.383905 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:44.384607 (+ 702us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:44.384649 (+ 42us) server_negotiation.cc:299] Negotiation successful
0810 14:57:44.384758 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 14:57:44.385576 23677 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3ab960de4754f24a1dffcfa67427f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } } }
I20260810 14:57:44.385699 23647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.372629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53882 (local address 127.30.74.124:44539)
0810 14:57:44.372684 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.372686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.373327 (+ 641us) server_negotiation.cc:407] Connection header received
0810 14:57:44.373431 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.373432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.373445 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.373477 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.374957 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.375627 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.378643 (+ 3016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.378957 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.379003 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.381155 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:44.381156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:44.381157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:44.381165 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:44.383003 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.383714 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.383716 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.383716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.383787 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.384336 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.384351 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.384352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.384483 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:44.384580 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:44.385529 (+ 949us) server_negotiation.cc:299] Negotiation successful
0810 14:57:44.385615 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 14:57:44.385720 23677 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:44.385891 23685 mvcc.cc:205] Tried to move back new op lower bound from 7316987348438114304 to 7316987348429959168. Current Snapshot: MvccSnapshot[applied={T|T < 7316987348438114304}]
I20260810 14:57:44.386679 23555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759: Bootstrap starting.
I20260810 14:57:44.387542 23555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759: Neither blocks nor log segments found. Creating new log.
I20260810 14:57:44.387900 23555 log.cc:824] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759: Log is configured to *not* fsync() on all Append() calls
I20260810 14:57:44.388974 23663 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3ab960de4754f24a1dffcfa67427f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } } }
I20260810 14:57:44.389074 23663 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:57:44.389406 23686 mvcc.cc:205] Tried to move back new op lower bound from 7316987348438114304 to 7316987348429959168. Current Snapshot: MvccSnapshot[applied={T|T < 7316987348438114304}]
I20260810 14:57:44.390651 23555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759: No bootstrap required, opened a new log
I20260810 14:57:44.391438 23683 catalog_manager.cc:1347] Generated new cluster ID: ce5181b4d0c446a2a82429350e3842ad
I20260810 14:57:44.391486 23683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:57:44.392026 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.373147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.30.74.125:34031)
0810 14:57:44.373202 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.373204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.380430 (+ 7226us) server_negotiation.cc:407] Connection header received
0810 14:57:44.380540 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.380541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.380553 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.380582 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.383068 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.383772 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.385907 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.386289 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.386338 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.388392 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:44.388393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:44.388394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:44.388401 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:44.390647 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.391087 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.391089 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.391089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.391165 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.391418 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.391419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.391419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.391558 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:44.391639 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:44.391864 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 14:57:44.391942 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 14:57:44.392443 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:44.372948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43198 (local address 127.30.74.126:45157)
0810 14:57:44.372996 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.372998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.373323 (+ 325us) server_negotiation.cc:407] Connection header received
0810 14:57:44.373429 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.373430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.373443 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.373475 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.381265 (+ 7790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.382002 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.385128 (+ 3126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:44.385411 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:44.385461 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:44.389329 (+ 3868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:57:44.389331 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:57:44.389331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:57:44.389341 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:57:44.391046 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.391517 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.391519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.391519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.391582 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:57:44.391807 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:57:44.391808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:57:44.391808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:57:44.391937 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:57:44.392013 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:44.392239 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 14:57:44.392354 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 14:57:44.393055 23555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } }
I20260810 14:57:44.393280 23555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c4aba9989f34481a2ac08c88e8a2759, State: Initialized, Role: FOLLOWER
I20260810 14:57:44.393373 23555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } }
I20260810 14:57:44.393622 23695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } } }
I20260810 14:57:44.393692 23555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:57:44.393699 23695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 14:57:44.395560 23706 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:57:44.395612 23706 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:57:44.395701 23525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:57:44.396739 23525 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759 [term 1 FOLLOWER]: Refusing update from remote peer b3ab960de4754f24a1dffcfa67427f53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:57:44.397127 23649 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:57:44.397110 23526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b3ab960de4754f24a1dffcfa67427f53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4c4aba9989f34481a2ac08c88e8a2759"
I20260810 14:57:44.397974 23526 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ab960de4754f24a1dffcfa67427f53 in term 1.
I20260810 14:57:44.399164 23695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3ab960de4754f24a1dffcfa67427f53. Latest consensus state: current_term: 1 leader_uuid: "b3ab960de4754f24a1dffcfa67427f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } } }
I20260810 14:57:44.399274 23695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:57:44.399317 23695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3ab960de4754f24a1dffcfa67427f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } } }
I20260810 14:57:44.399363 23695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:57:44.410432 23649 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3ab960de4754f24a1dffcfa67427f53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3ab960de4754f24a1dffcfa67427f53" member_type: VOTER last_known_addr { host: "127.30.74.126" port: 45157 } } peers { permanent_uuid: "4c4aba9989f34481a2ac08c88e8a2759" member_type: VOTER last_known_addr { host: "127.30.74.125" port: 34031 } } peers { permanent_uuid: "b5a8b09cc9124c9c904306226f6c2b62" member_type: VOTER last_known_addr { host: "127.30.74.124" port: 44539 } } }
I20260810 14:57:44.410574 23649 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53 [sys.catalog]: This master's current role is: LEADER
I20260810 14:57:44.413254 23683 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:57:44.414026 23683 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:57:44.423940 23683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3ab960de4754f24a1dffcfa67427f53: Generated new TSK 0
I20260810 14:57:44.424204 23683 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 14:57:44.453449 23559 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:44.451967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46708 (local address 127.30.74.125:34031)
0810 14:57:44.452016 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.452018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.452738 (+ 720us) server_negotiation.cc:407] Connection header received
0810 14:57:44.452854 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.452855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.452868 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.452900 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.453374 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46708: BlockingRecv error: recv got EOF from 127.0.0.1:46708 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 14:57:44.454244 23475 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:44.452418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43202 (local address 127.30.74.126:45157)
0810 14:57:44.452459 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.452461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.453717 (+ 1256us) server_negotiation.cc:407] Connection header received
0810 14:57:44.453819 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.453821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.453832 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.453861 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.454176 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43202: BlockingRecv error: recv got EOF from 127.0.0.1:43202 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 14:57:44.455320 23645 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:44.454608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53902 (local address 127.30.74.124:44539)
0810 14:57:44.454662 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.454664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.454781 (+ 117us) server_negotiation.cc:407] Connection header received
0810 14:57:44.454879 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.454880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.454890 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.454920 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.455259 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53902: BlockingRecv error: recv got EOF from 127.0.0.1:53902 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 14:57:44.457717 23559 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:44.456888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46712 (local address 127.30.74.125:34031)
0810 14:57:44.456926 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.456928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.457180 (+ 252us) server_negotiation.cc:407] Connection header received
0810 14:57:44.457291 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.457292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.457310 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.457340 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.457667 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46712: BlockingRecv error: recv got EOF from 127.0.0.1:46712 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 14:57:44.460780 23559 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:44.459936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46722 (local address 127.30.74.125:34031)
0810 14:57:44.459970 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.459972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.460226 (+ 254us) server_negotiation.cc:407] Connection header received
0810 14:57:44.460361 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.460362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.460373 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.460402 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.460732 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46722: BlockingRecv error: recv got EOF from 127.0.0.1:46722 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 14:57:44.464700 23559 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:44.463864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46724 (local address 127.30.74.125:34031)
0810 14:57:44.463902 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.463904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.464138 (+ 234us) server_negotiation.cc:407] Connection header received
0810 14:57:44.464235 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.464236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.464246 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.464323 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.464639 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46724: BlockingRecv error: recv got EOF from 127.0.0.1:46724 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 14:57:44.470721 23559 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:44.469825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46740 (local address 127.30.74.125:34031)
0810 14:57:44.469867 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.469869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.470145 (+ 276us) server_negotiation.cc:407] Connection header received
0810 14:57:44.470253 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.470254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.470267 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.470297 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.470668 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46740: BlockingRecv error: recv got EOF from 127.0.0.1:46740 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 14:57:44.481191 23559 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:44.480095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46744 (local address 127.30.74.125:34031)
0810 14:57:44.480145 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.480147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.480500 (+ 353us) server_negotiation.cc:407] Connection header received
0810 14:57:44.480616 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.480617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.480631 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.480694 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.481115 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46744: BlockingRecv error: recv got EOF from 127.0.0.1:46744 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 14:57:44.499559 23559 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:44.498532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46748 (local address 127.30.74.125:34031)
0810 14:57:44.498581 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.498584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.498904 (+ 320us) server_negotiation.cc:407] Connection header received
0810 14:57:44.499016 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.499017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.499029 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.499061 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.499464 (+ 403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46748: BlockingRecv error: recv got EOF from 127.0.0.1:46748 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 14:57:44.534538 23559 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:44.533165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46756 (local address 127.30.74.125:34031)
0810 14:57:44.533384 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.533387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.533757 (+ 370us) server_negotiation.cc:407] Connection header received
0810 14:57:44.533869 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.533870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.533884 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.533916 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.534455 (+ 539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46756: BlockingRecv error: recv got EOF from 127.0.0.1:46756 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260810 14:57:44.600937 23559 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:44.600071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46772 (local address 127.30.74.125:34031)
0810 14:57:44.600171 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.600173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.600316 (+ 143us) server_negotiation.cc:407] Connection header received
0810 14:57:44.600414 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.600415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.600430 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.600474 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.600865 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46772: BlockingRecv error: recv got EOF from 127.0.0.1:46772 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260810 14:57:44.731196 23559 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:44.730346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46788 (local address 127.30.74.125:34031)
0810 14:57:44.730451 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.730454 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.730555 (+ 101us) server_negotiation.cc:407] Connection header received
0810 14:57:44.730659 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.730660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.730674 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.730754 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.731109 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46788: BlockingRecv error: recv got EOF from 127.0.0.1:46788 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 14:57:44.989421 23559 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:57:44.988707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46792 (local address 127.30.74.125:34031)
0810 14:57:44.988799 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:44.988801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:44.988813 (+ 12us) server_negotiation.cc:407] Connection header received
0810 14:57:44.988883 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:44.988884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:44.988898 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:44.988970 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:57:44.989348 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46792: BlockingRecv error: recv got EOF from 127.0.0.1:46792 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:57:45.357513 23675 catalog_manager.cc:1259] Loaded cluster ID: ce5181b4d0c446a2a82429350e3842ad
I20260810 14:57:45.357672 23675 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62: loading cluster ID for follower catalog manager: success
I20260810 14:57:45.358618 23675 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62: acquiring CA information for follower catalog manager: success
I20260810 14:57:45.358976 23675 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b5a8b09cc9124c9c904306226f6c2b62: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:57:45.396064 23706 catalog_manager.cc:1259] Loaded cluster ID: ce5181b4d0c446a2a82429350e3842ad
I20260810 14:57:45.396152 23706 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759: loading cluster ID for follower catalog manager: success
I20260810 14:57:45.397056 23706 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759: acquiring CA information for follower catalog manager: success
I20260810 14:57:45.397401 23706 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4c4aba9989f34481a2ac08c88e8a2759: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:57:45.505465 23780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:45.502760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46800 (local address 127.30.74.125:34031)
0810 14:57:45.502886 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:45.502888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:45.502918 (+ 30us) server_negotiation.cc:407] Connection header received
0810 14:57:45.502991 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:45.502999 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:45.503053 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:45.503136 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:45.503300 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:45.503805 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:45.504679 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:45.504882 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:45.504945 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:45.505115 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:45.505237 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:45.505299 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:45.505331 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 14:57:45.505351 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260810 14:57:45.509954 23787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:45.507544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53904 (local address 127.30.74.124:44539)
0810 14:57:45.507655 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:45.507657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:45.507664 (+ 7us) server_negotiation.cc:407] Connection header received
0810 14:57:45.507700 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:45.507701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:45.507735 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:45.507810 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:45.507948 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:45.508496 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:45.509228 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:45.509442 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:45.509499 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:45.509623 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:45.509724 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:45.509776 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:45.509814 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 14:57:45.509829 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":64,"threads_started":1}
I20260810 14:57:45.514969 23795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:45.511787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43206 (local address 127.30.74.126:45157)
0810 14:57:45.512133 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:45.512135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:45.512144 (+ 9us) server_negotiation.cc:407] Connection header received
0810 14:57:45.512190 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:45.512191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:45.512246 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:45.512358 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:45.512535 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:45.513025 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:45.513945 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:45.514167 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:45.514228 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:45.514463 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:45.514592 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:45.514668 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:45.514734 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 14:57:45.514761 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":73,"threads_started":1}
I20260810 14:57:45.515025 23780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:45.511732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46802 (local address 127.30.74.125:34031)
0810 14:57:45.511812 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:45.511814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:45.511917 (+ 103us) server_negotiation.cc:407] Connection header received
0810 14:57:45.511978 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:45.511979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:45.511997 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:45.512075 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:45.512178 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:45.512979 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:45.514158 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:45.514471 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:45.514523 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:45.514739 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:45.514772 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:45.514802 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:45.514925 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 14:57:45.514950 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 14:57:45.515604 23787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:57:45.511891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53912 (local address 127.30.74.124:44539)
0810 14:57:45.511957 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 14:57:45.511959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 14:57:45.512014 (+ 55us) server_negotiation.cc:407] Connection header received
0810 14:57:45.512081 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:57:45.512081 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:57:45.512093 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:57:45.512141 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:57:45.512332 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:45.513037 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:45.514556 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:57:45.514832 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:57:45.514914 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:57:45.515050 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:57:45.515108 (+ 58us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:57:45.515399 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0810 14:57:45.515494 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 14:57:45.515517 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 14:57:45.517275 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 23401
I20260810 14:57:45.525977 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 23479
I20260810 14:57:45.532733 31017 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task2SWuiv/build/release/bin/kudu with pid 23563
2026-08-10T14:57:45Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2921 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5873 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (279016 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST